19 Commits

Author SHA1 Message Date
dependabot[bot] 6b8c3ae6b0 chore(deps): bump the python group across 1 directory with 4 updates
Updates the requirements on [httpcore](https://github.com/encode/httpcore), [pydantic-settings](https://github.com/pydantic/pydantic-settings), [mcp](https://github.com/modelcontextprotocol/python-sdk) and [markitdown](https://github.com/microsoft/markitdown) to permit the latest version.

Updates `httpcore` to 1.0.9
- [Release notes](https://github.com/encode/httpcore/releases)
- [Changelog](https://github.com/encode/httpcore/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/httpcore/compare/1.0.0...1.0.9)

Updates `pydantic-settings` to 2.14.2
- [Release notes](https://github.com/pydantic/pydantic-settings/releases)
- [Commits](https://github.com/pydantic/pydantic-settings/compare/v2.14.1...v2.14.2)

Updates `mcp` to 2.0.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v0.2.0...v2.0.0)

Updates `markitdown` from 0.1.6 to 0.1.7
- [Release notes](https://github.com/microsoft/markitdown/releases)
- [Commits](https://github.com/microsoft/markitdown/compare/v0.1.6...v0.1.7)

---
updated-dependencies:
- dependency-name: httpcore
  dependency-version: 1.0.9
  dependency-type: direct:production
  dependency-group: python
- dependency-name: pydantic-settings
  dependency-version: 2.14.2
  dependency-type: direct:production
  dependency-group: python
- dependency-name: mcp
  dependency-version: 2.0.0
  dependency-type: direct:production
  dependency-group: python
- dependency-name: markitdown
  dependency-version: 0.1.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-30 17:36:13 +00:00
RaresKeY 25c9e735ef fix(email): open settings after OAuth callback (#5803) 2026-07-30 14:57:07 +01:00
RaresKeY 28c333e647 fix(email): preserve OAuth SMTP security (#5802) 2026-07-30 12:24:39 +01:00
Husam 84709a00d9 fix(llm): omit temperature for major-only Opus ids (claude-opus-5) (#5761)
The version pattern in _anthropic_rejects_temperature() required a minor
component, so major-only ids like `claude-opus-5` never matched and the
guard reported that the model accepts `temperature`. Anthropic rejects the
field outright on Opus 4.7+, so every such call returned HTTP 400 and the
stream aborted with zero tokens ("the model returned an empty response").

Make the minor optional and read a missing minor as `.0`. The major is also
capped at 1-2 digits with a no-trailing-digit lookahead, mirroring the
minor: once the minor is optional, a greedy major would swallow the date in
`claude-3-opus-20240229` and read it as version 20240229, dropping
temperature from a model that accepts it.

Fixes #5753

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
2026-07-30 11:30:00 +01:00
Husam 578312200a fix(markdown): restore extracted blocks verbatim so $& and $$ survive (#5768)
The placeholder-restore pass in mdToHtml put code, math, mermaid and
allowed-HTML blocks back with a string replacement, so String.replace read
`$&`, `` $` ``, `$'` and `$$` in the *replacement* as substitution patterns.
A fenced block containing them rendered corrupted: `$&` re-inserted the
placeholder (`perl -pe 's/world/$& again/'` became
`s/world/___CODE_BLOCK_0___amp; again/`), `` $` `` and `$'` spliced in the
surrounding document, and `$$` collapsed to a single `$`.

Pass a function replacer at all four sites, matching the inline-code site
below them, which was already fixed this way. A function's return value is
inserted verbatim with no `$` interpretation.

The inline-code comment claimed `echo $1` would be read as a back-reference;
with a string search value there are no capture groups, so `$1` is already
literal. Reworded to name the four sequences that do corrupt.

Fixes #5663
2026-07-30 10:48:31 +01:00
Husam f23221420f fix(skills): replace deprecated utcnow in skill timestamp helper (#5777)
* fix(skills): replace deprecated utcnow in skill timestamp helper

_now_iso() builds the 'created' value in skill frontmatter. datetime.utcnow()
returns a naive datetime and has been deprecated since Python 3.12, scheduled
for removal. Switch to the timezone-aware datetime.now(timezone.utc), keeping
the serialized YYYY-MM-DDTHH:MM:SSZ shape unchanged so existing skill files
keep parsing.

timezone.utc is used rather than the datetime.UTC alias, which is 3.11+ only.

Adds regression tests covering the deprecation, the serialized shape, and
UTC correctness under a non-UTC local timezone -- the last guards against a
bare datetime.now(), which yields the same shape but local wall time.

Fixes #5697

* test(skills): skip timezone mutation where unsupported

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-30 09:54:59 +01:00
holden093 6a84398e75 fix(skills): use utility model for skill tests instead of chat default (#5746)
Skill tests are background automation tasks (like auto-naming and
memory audit) and should use the configured utility model. Previously
they resolved via resolve_endpoint("default") which returned the
chat model, bypassing the utility model entirely.

This completes the sweep started in PR #4027 which fixed auto-naming
and memory audit but missed skill tests.
2026-07-30 09:06:31 +01:00
RaresKeY 3250a4ce68 fix(ci): clear review label when issues close (#5813)
The issue-close lifecycle change is narrowly scoped and correct. Closed issues remove the stale \`ready for review\` label and return before normal validation can restore it. Focused regressions cover closure and subsequent edits to a closed issue.

The branch was updated onto current \`dev\`. The focused test, merged-result validation, diff checks, and GitHub CI passed. No blocking review threads remain.
2026-07-29 22:04:28 +01:00
Boody cb0f6af002 Merge pull request #5822 from bitboody/tts_cache_fix
feat(tts): implement TTS cache size limit and eviction policy
2026-07-29 16:48:41 +03:00
Boody 9297bed5b9 add ODYSSEUS_TTS_CACHE_MAX_BYTES environment variable to docker-compose 2026-07-29 12:54:55 +03:00
Boody 2e631ad816 improve cache size calculation by filtering file types 2026-07-29 12:47:55 +03:00
Boody d183fe545b add test for cache eviction handling unlink errors gracefully 2026-07-29 12:42:20 +03:00
Boody 9914651cc9 improve cache eviction logic to handle file access errors and ensure stability 2026-07-29 12:41:31 +03:00
Boody 46905ab9b0 added ODYSSEUS_TTS_CACHE_MAX_BYTES env variable to docker compose files 2026-07-29 12:32:43 +03:00
Boody 61c138d9e7 fixed .env.example ODYSSEUS_TTS_CACHE_MAX_BYTES into correct 500 MBs 2026-07-29 12:26:09 +03:00
Tal.Yuan 25a4d134b1 refactor(routes): move search domain into routes/search/ subpackage (#5779)
Slice 2j of the route-domain reorganization (#4082/#4071). Moves
search_routes.py into routes/search/, leaving a backward-compat
sys.modules shim. Pure file reorganization, no behavior change.
2026-07-28 22:26:29 +02:00
Boody 98e4d8451b fix(tests): update environment variable for TTS cache limit to include ODYSSEUS prefix 2026-07-28 22:00:54 +03:00
Boody 5104a9a967 feat(tts): implement TTS cache size limit and eviction policy 2026-07-28 21:34:03 +03:00
RaresKeY 01790c2f08 fix(mcp): keep built-in servers on SDK v1 (#5820) 2026-07-28 18:11:34 +01:00
46 changed files with 1472 additions and 1454 deletions
+1
View File
@@ -189,6 +189,7 @@ SEARXNG_INSTANCE=http://localhost:8080
# ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=26214400 # email compose attachment (25 MB)
# ODYSSEUS_STT_MAX_AUDIO_BYTES=26214400 # speech-to-text audio (25 MB)
# ODYSSEUS_ICS_MAX_BYTES=10485760 # calendar .ics import (10 MB)
# ODYSSEUS_TTS_CACHE_MAX_BYTES=524288000 # TTS cache (500 MB)
# ============================================================
# Host Docker access (explicit opt-in)
+2 -2
View File
@@ -8,8 +8,8 @@ body:
value: |
**Before submitting:** search [open issues](https://github.com/odysseus-dev/odysseus/issues)
and [discussions](https://github.com/odysseus-dev/odysseus/discussions) first.
The [roadmap](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md) is directional rather than a complete backlog.
Feature requests that duplicate an existing issue or accepted proposal may be closed as duplicates.
Feature requests that duplicate [ROADMAP.md](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md)
or an existing open issue will be closed as duplicates.
If your idea needs community input before it becomes a concrete proposal,
start a [discussion](https://github.com/odysseus-dev/odysseus/discussions/categories/ideas) instead.
+10 -3
View File
@@ -153,6 +153,16 @@ module.exports = async ({ github, context, core }) => {
}
}
const LABEL_BAD = 'needs more info';
const LABEL_GOOD = 'ready for review';
// Closed issues are no longer awaiting review.
// This also prevents later edits to closed issues from restoring the label.
if (issue.state === 'closed') {
await dropLabel(LABEL_GOOD);
return;
}
// ── Find existing bot comment to update in-place ──────────────────────────
const MARKER = '<!-- issue-description-check -->';
const { data: comments } = await github.rest.issues.listComments({
@@ -160,9 +170,6 @@ module.exports = async ({ github, context, core }) => {
});
const existing = comments.find(c => c.user.type === 'Bot' && c.body.includes(MARKER));
const LABEL_BAD = 'needs more info';
const LABEL_GOOD = 'ready for review';
if (failures.length === 0) {
if (existing) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
@@ -2,7 +2,7 @@ name: ci / issue description check
on:
issues:
types: [opened, edited, reopened]
types: [opened, edited, reopened, closed]
permissions:
issues: write
+4 -8
View File
@@ -1,5 +1,3 @@
# Odysseus
<p align="center">
<img src="docs/odysseus-wordmark.png" alt="Odysseus" width="238">
</p>
@@ -11,8 +9,6 @@
<p align="center">
<a href="#quick-start">Quick Start</a> ·
<a href="docs/setup.md">Setup Guide</a> ·
<a href="docs/ARCHITECTURE.md">Architecture</a> ·
<a href="SECURITY.md">Security</a> ·
<a href="CONTRIBUTING.md">Contributing</a> ·
<a href="ROADMAP.md">Roadmap</a>
</p>
@@ -55,15 +51,15 @@ Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration
## Demo
Explore the interface through the [interactive product tour](docs/index.html).
A full hover-to-play tour lives on the landing page: [`docs/index.html`](docs/index.html).
## Contributing
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, documentation, and small focused refactors. Read the [contributing guide](CONTRIBUTING.md), review the [public roadmap](ROADMAP.md), and browse the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues).
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, docs, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).
## Security
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model or service ports publicly. Read the [security policy](SECURITY.md) and the [deployment security guidance](docs/setup.md#security-notes).
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Deployment details are in the [setup guide](docs/setup.md#security-notes).
## Star History
@@ -77,4 +73,4 @@ Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled
## License
Licensed under AGPL-3.0-or-later. See the [license](LICENSE) and [acknowledgments](ACKNOWLEDGMENTS.md).
AGPL-3.0-or-later -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
+75 -43
View File
@@ -1,55 +1,87 @@
# Roadmap
# Roadmap / Help Wanted
This document provides a high-level view of the areas Odysseus is currently improving.
Odysseus is on a voyage, but not home yet. It works great for me (lol), but this ship is moving fast and feedback/help would be appreciated! (I don't know what I'm doing, help).
It is directional rather than exhaustive. Priorities may change as the project evolves, defects are discovered, and maintainers learn more from implementation work and user feedback.
If you see weird CSS, strange layout behavior, or a suspiciously murky corner of
the codebase, you are probably right to stay away.
For current implementation work, see the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues). Accepted behaviour should be documented in the repository alongside the code.
## High Priority
## Current priorities
- SQUASH BUGS
- Fresh install smoke tests on Linux, macOS, and Windows. Docker, native Python,
and WSL all need coverage.
### Reliability and setup
- Integration audit: do integrations even work? Confirm what works, what needs setup docs, and what should be removed or hidden.
- Cookbook reliability on other computers. This is probably the area most likely to need work across different machines, GPUs, drivers, shells, and Python environments.
- Cookbook SGLang support across platforms. Make sure SGLang setup/serve works
predictably on Linux, Windows/WSL, macOS where possible, Docker, and common
NVIDIA/AMD hardware paths.
- Deep Research model presets by hardware. Recommend approved model/parameter
profiles for small, medium, and large local setups so people with different
hardware can use Deep Research without guessing. Surface this either in Deep
Research settings or as a Cookbook scan/dropdown suggestion.
- Cookbook model scan/download ranking. Prioritize newer architectures and
better hardware-fit models instead of scoring everything almost the same.
Ranking should account for architecture age, quant format, VRAM/RAM fit,
backend support, vision/mmproj requirements, and likely serve reliability.
- Cookbook error feedback and logging. Failed downloads, dependency installs,
preflights, and serve jobs should show the actual command/output/error in the
UI, with copyable logs and clear next steps instead of just "crashed".
- Agent prompt/context bloat. Agent mode is too heavy for smaller local models:
tool schemas, skills, memory, documents, and instructions can eat the context
before the user request really starts. We need slimmer prompts, better tool
selection, smaller default tool sets, and clearer guidance for models with
4k/8k/16k context windows.
- Local model speculative decoding support. For Odysseus-tuned local models,
plan to ship or recommend a small same-tokenizer draft model when the serving
backend supports it. Early vLLM testing showed a generic `Qwen3-0.6B` draft
beside `Qwen3-8B` can materially reduce wall time, while an unsupported
DSpark conversion performed poorly. Treat this as a supported draft-model lane
first; keep MTP-specific packaging as future work only when the architecture
and runtime support are real. Judge this by time-to-success, tool correctness,
grammar, and unchanged target output, not tokens/sec alone.
- Skill/tool prompt-injection audit. User-editable skills, notes, documents,
fetched pages, and memories should be treated as untrusted data. Keep testing
whether models follow malicious instructions from those surfaces.
- Better degraded-state reporting for ChromaDB, SearXNG, email, ntfy, and provider probes.
- Email performance audit. Fetching, searching, opening, deleting, and sending
email can feel slow, especially over IMAP/SMTP providers with high latency.
Need someone who knows mail performance to profile the current flow, identify
whether the bottleneck is IMAP folder select/fetch, cache invalidation,
attachment/body loading, SMTP handshakes, or frontend refresh behavior, then
propose safer caching/prefetch/batching without breaking multi-account state.
- Provider setup/probing audit for Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, and DeepSeek.
- Improve fresh-install and smoke-test coverage across supported environments.
- Make provider setup, probing, and failure states more predictable.
- Improve Cookbook reliability across hardware, operating systems, drivers, shells, and serving backends.
- Improve degraded-state reporting and recovery guidance when optional services are unavailable.
## Refactor Targets
- CSS cleanup. `static/style.css` basically Calypso's island atm.
- Tour core helper. The onboarding tours have too much copy-pasted scaffolding; promote a shared `tour-core.js` helper before adding more tours.
- Modal/window positioning cleanup. Some window controls have improved, but the
underlying popup/dropdown/fixed-position behavior is still too fragile.
- Mobile media override discoverability. A lot of "CSS did not move" bugs are mobile `@media` overrides of the same selector; comments or linting around desktop/mobile paired rules would help.
- Dead code pass for old routes, stale feature flags, and unused UI states.
### Local model workflows
## Frontend
- Improve hardware-aware model recommendations and compatibility guidance.
- Evaluate serving optimizations, including speculative decoding, through reproducible benchmarks.
- Improve installation, preflight checks, logging, and error reporting for local model serving.
- Reduce prompt and context overhead for smaller local models.
- Expand the Editor for quicker, more robust everyday use. Better file/document
handling, smoother window behavior, clearer save/export flows, stronger image
editing affordances, and fewer brittle edge cases.
- Better AI integration for Notes and Todos. Notes should be easier for the
agent to read, update, summarize, and turn into actions. Todos should be
assignable to an agent from the UI, possibly through a button, task action,
or dedicated skill/tool flow.
- Mobile gallery/editor polish. Easier to launch/download inpaint model or any missing pieces.
- Accessibility pass: keyboard navigation, focus states, contrast, reduced motion.
- Improve empty states and error messages on fresh installs.
- Tighten first-run setup, hints, and tours so they do not repeat or fight each other.
- Vendor CDN assets eventually for a more fully self-hosted/offline mode.
### Safety and resilience
## Backend
- Continue hardening tool execution, filesystem access, credentials, networking, and destructive operations.
- Treat content from documents, notes, memories, skills, and fetched pages as potentially untrusted.
- Improve security-focused regression coverage and operational guidance.
- Review integrations that expand access to sensitive data or privileged operations.
- More tests around endpoint probing and provider setup.
- Better task scheduler defaults and visibility.
- Backup/restore guide and helper flow for `data/`.
- Security hardening around admin-only tools and clear docs for their risk.
### Product usability
## Not The Focus Right Now
- Improve first-run setup, onboarding, hints, and tours.
- Improve accessibility, keyboard navigation, focus behaviour, contrast, and reduced-motion support.
- Improve empty states, error messages, and recovery paths.
- Strengthen Notes, Todos, Editor, mobile, and everyday workspace flows.
### Architecture and maintainability
- Reduce duplication and technical debt through focused, reviewable refactors.
- Improve subsystem documentation as behaviour and architecture become stable.
- Remove stale code, obsolete feature flags, and unsupported integrations.
- Keep implementation decisions grounded in current code and verified behaviour.
## Tracking work
Concrete implementation tasks, defects, proposals, and technical investigations are tracked in:
- [GitHub Issues](https://github.com/odysseus-dev/odysseus/issues)
- [Contributing Guide](CONTRIBUTING.md)
Maintainers may use additional private coordination tools for ownership, planning, and unresolved decisions.
This roadmap is not a complete backlog or a guarantee that a particular item will be delivered.
I prob shouldnt add more themes.
+1 -3
View File
@@ -37,6 +37,4 @@ Only `.env.example`, docs, source, tests, and static assets should be committed.
## Reporting
Report security vulnerabilities privately through [GitHub Security Advisories](https://github.com/odysseus-dev/odysseus/security/advisories/new).
Do not open a public issue or discussion, and do not disclose exploit details publicly.
Please report vulnerabilities privately via GitHub security advisories if available, or by opening a minimal issue that does not disclose exploit details.
+5 -5
View File
@@ -68,14 +68,14 @@ External content that reaches the LLM is treated as untrusted via `src/prompt_se
- `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer` everywhere.
- **CSP:** nonce-based `script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net`. `style-src 'unsafe-inline'` is intentionally kept — `static/index.html` ships inline `<style>` blocks and JS modules set `style=""` attributes at runtime. Inline styles do not execute script so the risk is visual-only. Removing this requires templating the HTML files and auditing all JS-set style attributes.
## Token-Supplied Model Endpoints
Direct `/api/v1/chat` requests with a token-supplied `base_url` must use a public HTTP(S) endpoint. This restriction applies only to untrusted direct values; administrator-configured endpoints may intentionally use local or LAN URLs for private model providers.
## Known Gaps
These are open, acknowledged, and contributor help is welcome:
1. **No shell/filesystem sandbox.** The agent `bash` and `read_file`/`write_file` tools run as the app process user with no network egress filtering or filesystem confinement. A successful prompt-injection reaching a shell-enabled admin session can make outbound requests to internal services. See #1058 for the sandbox proposal.
2. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
2. **SSRF via `/api/v1/chat` `base_url` parameter.** A chat-scoped API token can supply an arbitrary `base_url`; the server forwards the LLM request to that host without validating the scheme or address. PR #1039 fixes this.
3. **`src/search/` partial consolidation.** `src.search.core` and `src.search.providers` correctly alias `services.search` via `sys.modules` replacement. `analytics`, `cache`, `content`, `query`, and `ranking` are still independent copies that can drift. The SSRF regression tests in `tests/test_webhook_ssrf_resilience.py` test `src.webhook_manager` directly (separate from search), so the safety net there is intact. See #1058.
4. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
+1 -1
View File
@@ -692,7 +692,7 @@ from routes.history.history_routes import setup_history_routes
app.include_router(setup_history_routes(session_manager, upload_handler=upload_handler))
# Search
from routes.search_routes import setup_search_routes
from routes.search.search_routes import setup_search_routes
app.include_router(setup_search_routes(config))
# Presets
-34
View File
@@ -1,34 +0,0 @@
# Odysseus discovery maps
Compact, code-grounded discovery maps of cross-cutting systems in the checked-in Odysseus codebase. They preserve investigation context and open factual questions; they are not canonical subsystem specifications, a feature certification, or a substitute for normal testing.
> [!IMPORTANT]
> Checked-in code is the source of truth for current behaviour. Mature subsystem specifications, where they exist, are the canonical documentation of accepted subsystem behaviour. Check code, tests, and configuration before reconciling a discovery finding. Discovery remains non-canonical.
## Explore the maps
| Document | Purpose |
|---|---|
| [Current system map](system-map.md) | Records evidence locations, confirmed local observations, and factual open questions about subsystem boundaries. |
| [Safety boundaries](safety-boundaries.md) | Records evidence about broad authority, safeguards, confirmed risks or gaps, and unverified behaviour. |
## Working rules
- **Trace the code first.** Confirm the current path in source before recording a claim.
- **Promote selectively.** When an owning mature specification exists, add a fact only when it is verified, useful, and not already represented there.
- **Record missing ownership.** When no owning specification exists, retain the verified finding in discovery and record missing documentation ownership as a follow-up.
- **Retain uncertainty here.** Keep unresolved questions and useful investigation context in discovery rather than treating them as canonical truth.
- **Keep specifications current-state only.** Do not record intentions, design direction, refactor plans, decision history, priority, ownership, or sequencing here.
- **Investigate with cause.** Do not exhaustively revalidate existing functionality without a report, visible failure, relevant change, or high-authority review need.
- **Review authority carefully.** Give execution, data access, external tools, credentials, destructive operations, and unattended work focused review.
- **Use stable locations.** Cite modules, routes, classes, and functions instead of fragile line ranges or generated evidence tables.
## Reconciliation flow
1. Start with the relevant map and trace the cited code.
2. Classify the finding against current source evidence and an owning mature specification where one exists.
3. Promote only verified, useful facts that are missing from an existing owning specification.
4. When no owning specification exists, retain the verified finding here and record missing documentation ownership as a follow-up; otherwise retain unresolved context here and correct stale wording.
> [!NOTE]
> This package intentionally contains no generator, validator, maturity scale, feature database, or parallel work tracker. The [architecture runtime inventory](./architecture-runtime-inventory.md) preserves dated structural metrics, investigation context, and historical planning as an explicitly non-canonical snapshot.
-331
View File
@@ -1,331 +0,0 @@
# Architecture runtime inventory
> [!WARNING]
> This document is a dated structural snapshot, not a canonical runtime specification.
> Counts, paths, and implementation details may drift as the repository changes.
> Verify implementation-sensitive claims against the current code and tests.
- **Branch:** `discovery`
- **Commit:** `c762efe1c97a`
- **Generated:** `2026-07-28T05:38:14+01:00`
- **Historical context:** readability and refactor planning in [#4071](https://github.com/odysseus-dev/odysseus/issues/4071) and [#4082](https://github.com/odysseus-dev/odysseus/issues/4082)
## Disposition
> [!NOTE]
> Reviewed for documentation classification. Stable runtime structure and
> subsystem ownership have been transferred to the proposed canonical
> destination, [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md), for
> maintainer review.
>
> This document retains dated metrics, rankings, investigation context,
> refactor-sensitive observations, and historical planning. Those contents are
> non-canonical and belong under `discovery/`.
| Content | Authority and destination |
|---|---|
| Stable runtime structure | Proposed canonical destination: `docs/ARCHITECTURE.md` |
| Stable subsystem boundaries | Proposed canonical destination: `docs/ARCHITECTURE.md` |
| Frontend module organization | `static/js/MODULE_SUMMARY.md` |
| Counts, line totals, and rankings | This non-canonical inventory |
| Investigation context and open questions | `discovery/` |
| Refactor options and prioritization | Issues, Plane, or non-canonical discovery material |
The transfer preserves the stable facts without promoting generated metrics or
historical prioritization into canonical documentation.
## Purpose
This inventory provides a reviewable map of the current repository structure,
large runtime modules, major subsystem boundaries, and refactor-sensitive areas.
It does not:
- define accepted subsystem behaviour;
- certify runtime correctness;
- prescribe a committed refactor sequence;
- replace focused specifications, tests, or source review.
For cross-cutting implementation evidence, see the [discovery maps](./README.md).
## Top-level runtime structure
| Area | Role |
|---|---|
| `app.py` | FastAPI application composition and entry point |
| `launcher.py` | Application launch support |
| `setup.py` | Native setup workflow |
| `core/` | Authentication, middleware, persistence, sessions, and platform primitives |
| `routes/` | HTTP and API route handlers |
| `src/` | Application services, orchestration, tools, providers, and runtime helpers |
| `services/` | Domain-oriented service packages |
| `mcp_servers/` | Built-in MCP server implementations |
| `scripts/` | CLI tools, diagnostics, maintenance, and migration helpers |
| `static/` | No-build browser frontend and bundled assets |
| `tests/` | Automated test suite and supporting test infrastructure |
## Directory snapshot
| Directory | Tracked files | Tracked Python files | Direct subdirectories |
|---|---:|---:|---|
| `src/` | 143 | 143 | `agent_tools/`, `model_capability_readers/`, `search/`, `tools/` |
| `routes/` | 73 | 73 | `admin_wipe/`, `cleanup/`, `compare/`, `contacts/`, `gallery/`, `history/`, `memory/`, `note/`, `research/` |
| `core/` | 11 | 11 | None |
| `services/` | 42 | 40 | `docs/`, `faces/`, `hwfit/`, `memory/`, `research/`, `search/`, `shell/`, `stt/`, `tts/`, `youtube/` |
| `mcp_servers/` | 5 | 5 | None |
| `scripts/` | 44 | 17 | `_completion/`, `_lib/`, `demo_email/` |
| `static/js/` | 154 | 0 | `calendar/`, `color/`, `compare/`, `editor/`, `emailLibrary/`, `markdown/`, `model/`, `research/`, `util/` |
| `tests/` | 768 | 758 | `cli/`, `helpers/`, `streaming/`, `tools/` |
> [!NOTE]
> Counts in this table use `git ls-files`, so generated caches, virtual
> environments, and other untracked local files are excluded.
## Largest backend modules
Large files are review signals, not proof that a module should be split.
Coupling, ownership, import compatibility, tests, and runtime authority matter more
than line count alone.
| Rank | File | Lines | Classes | Top-level functions | Review signal |
|---:|---|---:|---:|---:|---|
| 1 | `routes/email_routes.py` | 6032 | 1 | 58 | High |
| 2 | `src/agent_loop.py` | 5248 | 0 | 63 | High |
| 3 | `routes/cookbook_routes.py` | 4545 | 0 | 16 | High |
| 4 | `mcp_servers/email_server.py` | 2920 | 0 | 77 | High |
| 5 | `src/llm_core.py` | 2895 | 3 | 85 | High |
| 6 | `src/builtin_actions.py` | 2845 | 2 | 27 | High |
| 7 | `routes/model_routes.py` | 2743 | 0 | 65 | High |
| 8 | `src/task_scheduler.py` | 2627 | 1 | 8 | Medium |
| 9 | `core/database.py` | 2562 | 28 | 67 | High |
| 10 | `routes/gallery/gallery_routes.py` | 2325 | 0 | 16 | Medium |
| 11 | `routes/chat_routes.py` | 2063 | 0 | 18 | Medium |
| 12 | `routes/shell_routes.py` | 1971 | 1 | 21 | Medium |
| 13 | `src/visual_report.py` | 1933 | 0 | 11 | Medium |
| 14 | `routes/email_helpers.py` | 1888 | 3 | 48 | Medium |
| 15 | `routes/document_routes.py` | 1810 | 0 | 5 | Medium |
| 16 | `src/tools/cookbook.py` | 1705 | 0 | 34 | Medium |
| 17 | `routes/calendar_routes.py` | 1667 | 2 | 19 | Medium |
| 18 | `routes/skills_routes.py` | 1662 | 3 | 19 | Medium |
| 19 | `src/tool_schemas.py` | 1595 | 0 | 3 | Medium |
| 20 | `routes/email_pollers.py` | 1551 | 0 | 23 | Medium |
The largest current backend concentrations include:
- email routing and helper logic;
- agent-loop orchestration;
- Cookbook lifecycle and serving logic;
- provider and model routing;
- task scheduling;
- shared database models and persistence helpers.
These areas require focused ownership and compatibility analysis before structural
changes are attempted.
## Largest frontend modules
| Rank | File | Lines |
|---:|---|---:|
| 1 | `static/style.css` | 41132 |
| 2 | `static/js/document.js` | 11200 |
| 3 | `static/js/emailLibrary.js` | 8505 |
| 4 | `static/js/slashCommands.js` | 6520 |
| 5 | `static/js/chat.js` | 6001 |
| 6 | `static/js/settings.js` | 5819 |
| 7 | `static/js/notes.js` | 5365 |
| 8 | `static/app.js` | 4681 |
| 9 | `static/js/cookbookRunning.js` | 4433 |
| 10 | `static/js/galleryEditor.js` | 4386 |
| 11 | `static/js/cookbookServe.js` | 4305 |
| 12 | `static/js/calendar.js` | 3722 |
| 13 | `static/js/cookbook.js` | 3677 |
| 14 | `static/js/sessions.js` | 3665 |
| 15 | `static/js/documentLibrary.js` | 3422 |
| 16 | `static/js/tasks.js` | 3187 |
| 17 | `static/js/admin.js` | 3144 |
| 18 | `static/js/gallery.js` | 2958 |
| 19 | `static/js/cookbook-hwfit.js` | 2826 |
| 20 | `static/js/chatRenderer.js` | 2808 |
The browser frontend remains a no-build ES-module application. Its current source
tree is authoritative; the maintained structural summary is available in
[`static/js/MODULE_SUMMARY.md`](../static/js/MODULE_SUMMARY.md).
CSS modularization remains tracked separately in
[#2617](https://github.com/odysseus-dev/odysseus/issues/2617).
## Major subsystem boundaries
| Subsystem | Primary implementation locations |
|---|---|
| Application startup | `app.py`, `src/app_initializer.py`, `core/` |
| Authentication and sessions | `core/auth.py`, `core/middleware.py`, `core/session_manager.py`, `routes/auth_routes.py` |
| Chat and streaming | `routes/chat_routes.py`, `routes/chat_helpers.py`, `src/chat_handler.py`, `src/chat_processor.py`, `src/llm_core.py` |
| Agents and tools | `src/agent_loop.py`, `src/tool_execution.py`, `src/agent_tools/`, `src/tools/`, `src/tool_policy.py`, `src/tool_security.py` |
| Models and providers | `routes/model_routes.py`, `src/model_discovery.py`, `src/model_capabilities.py`, `src/endpoint_resolver.py`, `src/llm_core.py` |
| Cookbook and hardware fit | `routes/cookbook_routes.py`, `routes/cookbook_helpers.py`, `src/cookbook_serve_lifecycle.py`, `services/hwfit/` |
| Search and research | `routes/search_routes.py`, `services/search/`, `routes/research/`, `services/research/`, `src/deep_research.py` |
| Documents and retrieval | `routes/document_routes.py`, `src/document_processor.py`, `src/personal_docs.py`, `src/rag_manager.py`, `src/pdf_runtime.py` |
| Memory and skills | `routes/memory/`, `services/memory/`, `routes/skills_routes.py` |
| Email | `routes/email_routes.py`, `routes/email_helpers.py`, `routes/email_pollers.py`, `mcp_servers/email_server.py` |
| Calendar, contacts, notes, and tasks | `routes/calendar_routes.py`, `routes/contacts/`, `routes/note/`, `routes/task_routes.py`, `src/task_scheduler.py` |
| Media and speech | `routes/gallery/`, `routes/stt_routes.py`, `routes/tts_routes.py`, `services/stt/`, `services/tts/` |
| Persistence and operations | `core/database.py`, `src/runtime_paths.py`, `src/bg_jobs.py`, `routes/backup_routes.py`, `routes/cleanup/` |
For a broader evidence map, see
[`system-map.md`](./system-map.md).
## Refactor-sensitive areas
### Shared persistence
`core/database.py` is a central dependency containing models and shared persistence
helpers. Changes can affect routes, services, background work, tests, migrations,
and import compatibility.
A split should not begin from file size alone. It requires:
- an importer inventory;
- model and helper ownership decisions;
- migration compatibility checks;
- stable re-export or migration strategy;
- focused and full-suite validation.
### Agent orchestration
`src/agent_loop.py` coordinates model interaction, tool selection, policy decisions,
multi-round execution, and background behaviour. Extraction work must preserve tool
event semantics, policy enforcement, cancellation, and test patch points.
Historical agent-loop modularization discussion is tracked in
[#3266](https://github.com/odysseus-dev/odysseus/issues/3266).
### Tool implementation boundaries
Tool implementation is no longer represented by one proposed future package alone.
Current responsibilities are distributed across:
- `src/tool_execution.py`;
- `src/tool_schemas.py`;
- `src/tool_index.py`;
- `src/tool_policy.py`;
- `src/tool_security.py`;
- `src/agent_tools/`;
- `src/tools/`;
- remaining compatibility surfaces such as `src/tool_implementations.py`.
Historical tool modularization work is tracked in
[#3629](https://github.com/odysseus-dev/odysseus/issues/3629).
### Route ownership
`routes/` now contains both flat modules and domain packages. Existing package
boundaries should be extended only through focused changes. Broad mechanical route
movement would affect registration, imports, tests, monkeypatch targets, and
compatibility paths.
### Frontend concentration
The no-build frontend contains several large JavaScript modules and one central CSS
file. Refactors should preserve module load order, global compatibility exports,
DOM contracts, deep-link handling, and browser behaviour.
## Non-implemented architecture options
> [!NOTE]
> The paths below are historical or possible design directions. They do not describe
> the current repository and are not approved implementation plans.
Earlier planning discussed:
- renaming `app.py` to `main.py`;
- moving agent orchestration into a new `src/agent/` package;
- introducing broad `src/domain/`, `src/infra/`, `src/api/`, or `src/pkg/` layers;
- moving all routes into domain subpackages;
- splitting database models into a new infrastructure hierarchy.
These options should be reconsidered against the current tree rather than copied
forward as assumed targets.
## Refactor guardrails
- Keep structural changes behaviour-preserving.
- Change one ownership boundary at a time.
- Do not mix file movement with unrelated feature work.
- Preserve existing import and monkeypatch paths where compatibility is required.
- Identify focused tests before modifying high-authority modules.
- Validate startup, imports, and affected runtime paths.
- Avoid repository-wide package reorganizations without maintainer agreement.
- Treat generated metrics as snapshots, not architectural decisions.
## Reproduce the snapshot
Run these commands from the repository root.
```bash
# Tracked directory totals
for dir in src routes core services mcp_servers scripts static/js tests; do
files="$(git ls-files "$dir" | wc -l)"
python_files="$(git ls-files "$dir" '*.py' | wc -l)"
printf '%-14s tracked=%-5s python=%-5s\n' \
"$dir" \
"$files" \
"$python_files"
done
# Largest tracked backend files
git ls-files \
'app.py' \
'launcher.py' \
'setup.py' \
'core/*.py' \
'core/**/*.py' \
'routes/*.py' \
'routes/**/*.py' \
'services/*.py' \
'services/**/*.py' \
'src/*.py' \
'src/**/*.py' \
'mcp_servers/*.py' \
'scripts/*.py' \
'scripts/**/*.py' |
xargs wc -l |
sort -nr |
head -31
# Largest tracked frontend source files
git ls-files \
'static/*.js' \
'static/*.css' \
'static/*.html' \
'static/**/*.js' \
'static/**/*.css' \
'static/**/*.html' |
grep -vE '\.min\.js$' |
xargs wc -l |
sort -nr |
head -31
```
## Validation for architecture changes
Use the smallest relevant checks first, then expand according to risk:
```bash
python3 -m compileall -q app.py core routes services src
venv/bin/python -m pytest tests/<focused-test-file>.py -q
venv/bin/python -m pytest -q
```
Startup, browser, Docker, and integration checks may also be required depending on
the affected boundary.
## Related documentation
- [Documentation style](../docs/STYLE.md)
- [Discovery maps](../discovery/README.md)
- [Current system map](../discovery/system-map.md)
- [Safety boundaries](../discovery/safety-boundaries.md)
- [Frontend module summary](../static/js/MODULE_SUMMARY.md)
- [Testing standard](../tests/TESTING_STANDARD.md)
-142
View File
@@ -1,142 +0,0 @@
# Safety boundaries
> [!IMPORTANT]
> This non-canonical discovery map records code-grounded safeguards, confirmed risks or gaps, and unverified behaviour. Broad authority does not by itself establish a vulnerability. Verify the cited source before relying on a finding. No destructive test, external connection, or real credential was used for this map.
## Navigate the boundaries
- [Shell and subprocess execution](#shell-and-subprocess-execution)
- [Filesystem access and workspace confinement](#filesystem-access-and-workspace-confinement)
- [Agent-controlled tool dispatch](#agent-controlled-tool-dispatch)
- [MCP and external tool servers](#mcp-and-external-tool-servers)
- [Outbound network requests and URL validation](#outbound-network-requests-and-url-validation)
- [Secrets, credentials, and vault sessions](#secrets-credentials-and-vault-sessions)
- [Authentication and privileged administration](#authentication-and-privileged-administration)
- [Deletion, wipe, backup, and restore](#deletion-wipe-backup-and-restore)
- [Background jobs and unattended task execution](#background-jobs-and-unattended-task-execution)
## Shell and subprocess execution
- **Boundary:** Shell routes, agent `bash` and `python` tools, local model serving, and detached background jobs.
- **Available authority:** Commands run as the application process user and can create child processes.
- **User-controlled inputs:** Direct shell requests, model-produced tool arguments, scheduled-task prompts, and model-serving configuration.
- **Current safeguards:** Agent dispatch applies owner/admin checks and tool policy; process helpers use timeouts or bounded background-job lifecycle where implemented.
- **Confirmed risks or gaps:** Intentional authority with a confirmed gap: the agent shell starts in its workspace but is not sandboxed to it, and has no egress sandbox. This is documented in source and the threat model; it is not a newly demonstrated bypass.
- **Unverified behaviour:** Role-gate and disabled-tool outcomes, direct shell-route behaviour, and timeout, cancellation, and output handling for foreground and detached processes remain unverified.
## Filesystem access and workspace confinement
- **Boundary:** Agent read, write, patch, listing, glob, and grep tools.
- **Available authority:** Read and modify files within active workspace confinement or fallback allowlisted roots.
- **User-controlled inputs:** Tool paths, patches, file contents, search patterns, and workspace selection passed into the tool dispatcher.
- **Current safeguards:** [`src/tool_execution.py`](../src/tool_execution.py) resolves paths, blocks sensitive subpaths, applies allowlist containment, and tightens paths to the active workspace when one is bound. File tools use those resolvers.
- **Confirmed risks or gaps:** Intentional authority with safeguards. The file-tool policy does not sandbox the shell; treating a workspace as a whole-process containment boundary would be incorrect.
- **Unverified behaviour:** Traversal, symlink, sensitive-name, absolute-path, and workspace-switch behaviour remains unverified.
## Agent-controlled tool dispatch
- **Boundary:** Model output becomes native or parsed tool calls and is dispatched by the agent loop.
- **Available authority:** The authority of every enabled tool, including privileged built-ins and external tools.
- **User-controlled inputs:** Chat content, attached/retrieved content that may influence the model, tool arguments, per-request tool selection, and policy toggles.
- **Current safeguards:** [`src/tool_security.py`](../src/tool_security.py) blocks protected tools for non-admin users and fails closed for malformed tool names; [`src/tool_policy.py`](../src/tool_policy.py) supports disabled and guide-only policy; prompt-security helpers label untrusted context.
- **Confirmed risks or gaps:** Credible risk requiring verification: aliases, legacy text tools, native function calls, and MCP-qualified names must all reach the same policy outcome. The code has specific alias handling for email/MCP names, which makes this a sensitive compatibility seam.
- **Unverified behaviour:** The current policy outcomes for owner role, request mode, disabled state, native versus parsed invocation, qualified aliases, and external-content entry points remain unverified.
## MCP and external tool servers
- **Boundary:** Configured MCP servers and their tools are exposed to the agent through the MCP manager and routes.
- **Available authority:** Depends on the server: external network access, local process access, messaging, or data mutation may be delegated outside the application.
- **User-controlled inputs:** Server configuration, remote OAuth completion, tool arguments, and model-selected MCP calls.
- **Current safeguards:** MCP routes are registered through [`routes/mcp_routes.py`](../routes/mcp_routes.py); MCP-qualified tools are denied to non-admin users by [`src/tool_security.py`](../src/tool_security.py). OAuth state and token persistence are handled in [`src/mcp_oauth.py`](../src/mcp_oauth.py).
- **Confirmed risks or gaps:** Credible risk requiring verification: an MCP server authority is broader than the application can infer from its tool name. This map does not establish a trust or approval model for server installation and individual tool invocation.
- **Unverified behaviour:** Server onboarding, credential storage, server-origin trust, OAuth callback deployment, tool disablement, and invocation audit behaviour remain unverified.
## Outbound network requests and URL validation
- **Boundary:** Search/content fetch, research, webhooks, skill import, provider endpoints, and other HTTP clients.
- **Available authority:** The application can make outbound requests from its network position.
- **User-controlled inputs:** Search/fetch URLs, imported skill URLs, webhook configuration, and some endpoint settings.
- **Current safeguards:** [`src/url_security.py`](../src/url_security.py) validates untrusted public HTTP URLs and fails closed on unsuitable schemes or private addresses. [`services/search/content.py`](../services/search/content.py) resolves and rejects non-public hosts, pins resolved addresses for fetches, caps bodies, and limits redirects.
- **Confirmed risks or gaps:** Intentional split: administrator-created model endpoints may target private providers, while untrusted URLs use public-address checks. That distinction is required for self-hosted deployments but needs explicit call-site review.
- **Unverified behaviour:** The URL-source classification for outbound clients and the current handling of redirects and DNS changes remain unverified.
## Secrets, credentials, and vault sessions
- **Boundary:** Application-managed encrypted secrets, API keys, provider credentials, and Bitwarden/Vaultwarden CLI sessions.
- **Available authority:** Credentials unlock remote providers and connected personal services.
- **User-controlled inputs:** Administrative configuration, login/unlock requests, imported settings, and agent vault tool arguments.
- **Current safeguards:** [`src/secret_storage.py`](../src/secret_storage.py) uses a locally stored Fernet key with restrictive permissions for supported database secrets. Vault routes require an administrator, avoid passing master passwords in command arguments, and set restrictive permissions on the vault-session file.
- **Confirmed risks or gaps:** Confirmed current boundary: vault session data is persisted through the vault path, not through [`src/secret_storage.py`](../src/secret_storage.py). This is an unresolved question about current security semantics, not a confirmed exposure.
- **Unverified behaviour:** Current encryption-at-rest, owner scope, rotation, lock/logout, backup/restore, and log/tool-result exposure behaviour remains unverified.
## Authentication and privileged administration
- **Boundary:** Session authentication, API tokens, privileged routes, and internal tool loopback.
- **Available authority:** Administrative identity can access execution, settings, integrations, data deletion, and secrets.
- **User-controlled inputs:** Login/signup data, session cookies, API tokens, authentication configuration, and requests to privileged routes.
- **Current safeguards:** [`core/auth.py`](../core/auth.py), [`core/middleware.py`](../core/middleware.py), and route-level checks establish identity and administrator gates. [`app.py`](../app.py) warns when localhost bypass is configured; [`SECURITY.md`](../SECURITY.md) documents deployment requirements.
- **Confirmed risks or gaps:** Intentional authority with safeguards. Security depends on deployments keeping authentication enabled and internal services private; this map does not audit reverse-proxy or environment configuration.
- **Unverified behaviour:** Setup, anonymous, non-admin, admin, token, and internal-loopback behaviour, including privileged-route gate consistency, remains unverified.
## Deletion, wipe, backup, and restore
- **Boundary:** Administrative wipe, cleanup, backup import/export, and the backup restore command.
- **Available authority:** Delete or replace user data and credentials.
- **User-controlled inputs:** Administrative HTTP requests, cleanup choices, backup payloads, archive paths, and restore command options.
- **Current safeguards:** Administrative wipe routes use the administrative boundary. Cleanup exposes a preview route before mutation. The documented backup tool requires explicit restore confirmation, stages the old data directory, and validates archive members before extraction.
- **Confirmed risks or gaps:** Intentional destructive authority. Backup archives contain secrets by design, as documented in [`docs/backup-restore.md`](../docs/backup-restore.md); this is an operator confidentiality responsibility, not a code defect established here.
- **Unverified behaviour:** Role-gate, confirmation, archive-rejection, staged-recovery, and owner-isolation behaviour remains unverified. No destructive runtime test was performed.
## Background jobs and unattended task execution
- **Boundary:** Scheduled tasks, background-job monitor, startup tasks, and notification/delivery work that continue without an active browser request.
- **Available authority:** Scheduled agent work can obtain model access and, for eligible owners, shell and file tools; task output can interact with connected services.
- **User-controlled inputs:** Stored task prompt, schedule, model/crew selection, enabled-tool configuration, output target, and prior persisted state.
- **Current safeguards:** [`src/task_scheduler.py`](../src/task_scheduler.py) serializes execution, records task runs, associates work with an owner, and applies the agent owner-based tool gate. [`src/bg_jobs.py`](../src/bg_jobs.py) keeps bounded state and can terminate overlong subprocess jobs.
- **Confirmed risks or gaps:** Credible risk requiring verification: authority is inherited and exercised later, so changes to roles, task configuration, and disabled tools must be checked at execution time rather than assumed from task creation.
- **Unverified behaviour:** Creation, editing, role-change, scheduling, cancellation, restart-recovery, and execution behaviour remains unverified, including whether current policy is re-evaluated before privileged action.
-139
View File
@@ -1,139 +0,0 @@
# Current system map
> [!NOTE]
> This non-canonical discovery map is an evidence guide, not an exhaustive feature catalog or runtime certification. Verify the cited source before relying on a finding. Each section records local implementation observations, evidence locations, confirmed current problems, and unresolved factual questions.
## Navigate the system
- [Startup and application composition](#startup-and-application-composition)
- [Frontend shell and browser interaction](#frontend-shell-and-browser-interaction)
- [Chat, sessions, and streaming](#chat-sessions-and-streaming)
- [Agents, tools, and execution](#agents-tools-and-execution)
- [Models, providers, and local serving](#models-providers-and-local-serving)
- [Search and research](#search-and-research)
- [Documents, retrieval, and personal knowledge](#documents-retrieval-and-personal-knowledge)
- [Memory and skills](#memory-and-skills)
- [Email, calendar, contacts, notes, and tasks](#email-calendar-contacts-notes-and-tasks)
- [Media, speech, and image work](#media-speech-and-image-work)
- [Authentication, secrets, and privileged administration](#authentication-secrets-and-privileged-administration)
- [Persistence, background work, and operations](#persistence-background-work-and-operations)
## Startup and application composition
- **How it works:** [`app.py`](../app.py) creates the application, mounts static assets, constructs shared services, registers route factories, and owns lifespan startup and shutdown. [`src/app_initializer.py`](../src/app_initializer.py) prepares application state; [`core/`](../core/) provides persistence, authentication, middleware, sessions, and platform helpers.
- **Evidence locations:** [`app.py`](../app.py); [`src/app_initializer.py`](../src/app_initializer.py); [`core/database.py`](../core/database.py); [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/`](../routes/).
- **Known problems:** None recorded by this mapping.
- **Open question:** Which component currently owns startup and shutdown for each long-lived service?
## Frontend shell and browser interaction
- **How it works:** [`static/index.html`](../static/index.html) is served by the root and SPA deep-link routes in [`app.py`](../app.py); [`static/app.js`](../static/app.js), [`static/style.css`](../static/style.css), and [`static/js/`](../static/js/) implement the client surface.
- **Evidence locations:** [`static/index.html`](../static/index.html); [`static/app.js`](../static/app.js); [`static/js/`](../static/js/); [`static/style.css`](../static/style.css); [`app.py`](../app.py) deep-link handlers.
- **Known problems:** The `/backgrounds` route in [`app.py`](../app.py) calls `serve_html_with_nonce` for `static/backgrounds.html`, but that file is absent from [`static/`](../static/). This is a confirmed broken prototype route, not evidence about the rest of the frontend.
- **Open question:** Is `/backgrounds` currently an intentionally supported route or an obsolete prototype?
## Chat, sessions, and streaming
- **How it works:** [`routes/chat_routes.py`](../routes/chat_routes.py) and [`routes/chat_helpers.py`](../routes/chat_helpers.py) coordinate requests, session state, and SSE delivery. [`src/chat_handler.py`](../src/chat_handler.py), [`src/chat_processor.py`](../src/chat_processor.py), [`src/llm_core.py`](../src/llm_core.py), and [`src/session_actions.py`](../src/session_actions.py) provide message preparation, provider interaction, and session operations.
- **Evidence locations:** [`routes/chat_routes.py`](../routes/chat_routes.py); [`routes/chat_helpers.py`](../routes/chat_helpers.py); [`routes/session_routes.py`](../routes/session_routes.py); [`src/chat_handler.py`](../src/chat_handler.py); [`src/chat_processor.py`](../src/chat_processor.py); [`src/llm_core.py`](../src/llm_core.py); [`core/session_manager.py`](../core/session_manager.py).
- **Known problems:** [`src/agent_loop.py`](../src/agent_loop.py) annotates `_resolved_tool_event_name` with `Any` but imports no `Any` and does not enable postponed annotation evaluation. Python evaluates that annotation while importing the module, so this is an import-time defect at the checked baseline.
- **Open question:** No end-to-end provider or browser streaming run was performed for this map.
## Agents, tools, and execution
- **How it works:** [`src/agent_loop.py`](../src/agent_loop.py) drives multi-round tool use. [`src/tool_execution.py`](../src/tool_execution.py) dispatches calls and binds workspace context. [`src/agent_tools/`](../src/agent_tools/) contains individual implementations; [`src/tool_security.py`](../src/tool_security.py) and [`src/tool_policy.py`](../src/tool_policy.py) apply role and request policies. Long-running command work is represented by [`src/bg_jobs.py`](../src/bg_jobs.py).
- **Evidence locations:** [`src/agent_loop.py`](../src/agent_loop.py); [`src/tool_execution.py`](../src/tool_execution.py); [`src/agent_tools/`](../src/agent_tools/); [`src/tool_security.py`](../src/tool_security.py); [`src/tool_policy.py`](../src/tool_policy.py); [`src/tool_schemas.py`](../src/tool_schemas.py); [`src/bg_jobs.py`](../src/bg_jobs.py).
- **Known problems:** The import-time annotation defect above blocks the main agent/tool path. The shell is intentionally not a filesystem or network sandbox; that is an authority boundary, not by itself a vulnerability claim.
- **Open question:** Which native, legacy, and MCP-qualified invocation paths reach each policy gate?
## Models, providers, and local serving
- **How it works:** Model routes delegate to discovery, capabilities, endpoint resolution, and LLM core modules. Cookbook routes and hardware-fit services handle model lifecycle and local-serving support.
- **Evidence locations:** [`routes/model_routes.py`](../routes/model_routes.py); [`src/model_discovery.py`](../src/model_discovery.py); [`src/model_capabilities.py`](../src/model_capabilities.py); [`src/endpoint_resolver.py`](../src/endpoint_resolver.py); [`src/llm_core.py`](../src/llm_core.py); [`routes/cookbook_routes.py`](../routes/cookbook_routes.py); [`src/cookbook_serve_lifecycle.py`](../src/cookbook_serve_lifecycle.py); [`services/hwfit/`](../services/hwfit/).
- **Known problems:** None recorded by this mapping.
- **Open question:** Which endpoint inputs are administrator-created and permitted to use private provider addresses?
## Search and research
- **How it works:** HTTP search routes use [`services/search/`](../services/search/); research is exposed through [`routes/research/`](../routes/research/) and implemented in [`services/research/`](../services/research/), [`src/deep_research.py`](../src/deep_research.py), and related helpers. [`src/search/`](../src/search/) remains an import-compatibility layer for callers not yet moved to `services.search`.
- **Evidence locations:** [`routes/search_routes.py`](../routes/search_routes.py); [`services/search/`](../services/search/); [`routes/research/research_routes.py`](../routes/research/research_routes.py); [`services/research/`](../services/research/); [`src/deep_research.py`](../src/deep_research.py); [`src/search/`](../src/search/).
- **Known problems:** None recorded by this mapping.
- **Open question:** No live provider request was made; provider configuration and network access remain unverified.
## Documents, retrieval, and personal knowledge
- **How it works:** Document routes coordinate upload handling, document processing, and editor actions. Personal-document and RAG modules use Chroma and embedding clients. PDF viewing uses the optional-dependency loader in [`src/pdf_runtime.py`](../src/pdf_runtime.py); form extraction and filling live separately in [`src/pdf_forms.py`](../src/pdf_forms.py) and [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
- **Evidence locations:** [`routes/document_routes.py`](../routes/document_routes.py); [`src/upload_handler.py`](../src/upload_handler.py); [`src/document_processor.py`](../src/document_processor.py); [`src/document_actions.py`](../src/document_actions.py); [`src/personal_docs.py`](../src/personal_docs.py); [`src/rag_manager.py`](../src/rag_manager.py); [`src/embeddings.py`](../src/embeddings.py); [`src/pdf_runtime.py`](../src/pdf_runtime.py); [`src/pdf_forms.py`](../src/pdf_forms.py); [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
- **Known problems:** PDF viewing/runtime loading and PDF form processing are separate implementations. That separation is confirmed and intentional in the source; it is not a defect without a reported behavioural failure.
- **Open question:** Optional PDF dependencies and representative uploaded documents were not exercised.
## Memory and skills
- **How it works:** Memory routes use [`services/memory/`](../services/memory/) and vector helpers. Skills are exposed through [`routes/skills_routes.py`](../routes/skills_routes.py), stored and managed in [`services/memory/skills.py`](../services/memory/skills.py), and may be imported through [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
- **Evidence locations:** [`routes/memory/memory_routes.py`](../routes/memory/memory_routes.py); [`services/memory/`](../services/memory/); [`src/memory.py`](../src/memory.py); [`src/memory_vector.py`](../src/memory_vector.py); [`routes/skills_routes.py`](../routes/skills_routes.py); [`services/memory/skills.py`](../services/memory/skills.py); [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
- **Known problems:** None recorded by this mapping.
- **Open question:** Which imported skill content can reach execution-capable paths, and which validation occurs before that point?
## Email, calendar, contacts, notes, and tasks
- **How it works:** Dedicated route modules own email, CalDAV calendar, CardDAV contacts, notes, and tasks. Supporting modules include email helpers and pollers, CalDAV sync and writeback, and the task scheduler.
- **Evidence locations:** [`routes/email_routes.py`](../routes/email_routes.py); [`routes/calendar_routes.py`](../routes/calendar_routes.py); [`routes/contacts/contacts_routes.py`](../routes/contacts/contacts_routes.py); [`routes/note/note_routes.py`](../routes/note/note_routes.py); [`routes/task_routes.py`](../routes/task_routes.py); [`routes/assistant_routes.py`](../routes/assistant_routes.py); [`src/caldav_sync.py`](../src/caldav_sync.py); [`src/caldav_writeback.py`](../src/caldav_writeback.py); [`src/task_scheduler.py`](../src/task_scheduler.py).
- **Known problems:** None recorded by this mapping.
- **Open question:** External account behaviour, writeback, and delivery require controlled credentials and are not runtime-validated here.
## Media, speech, and image work
- **How it works:** Gallery and image routes coordinate media features. Service modules own speech and media integrations; [`src/generated_images.py`](../src/generated_images.py) and [`src/visual_report.py`](../src/visual_report.py) support artifact handling and presentation.
- **Evidence locations:** [`routes/gallery/gallery_routes.py`](../routes/gallery/gallery_routes.py); [`routes/stt_routes.py`](../routes/stt_routes.py); [`routes/tts_routes.py`](../routes/tts_routes.py); [`src/generated_images.py`](../src/generated_images.py); [`services/stt/`](../services/stt/); [`services/tts/`](../services/tts/); [`services/faces/`](../services/faces/); [`src/visual_report.py`](../src/visual_report.py).
- **Known problems:** None recorded by this mapping.
- **Open question:** Hardware- and provider-dependent media workflows were not exercised.
## Authentication, secrets, and privileged administration
- **How it works:** [`core/auth.py`](../core/auth.py) and [`core/middleware.py`](../core/middleware.py) provide identity and request gates. [`src/secret_storage.py`](../src/secret_storage.py) encrypts application-managed database secrets with a local Fernet key. Vault handling is separate: [`routes/vault_routes.py`](../routes/vault_routes.py) and [`src/tools/vault.py`](../src/tools/vault.py) invoke the Bitwarden CLI and persist its session data in the application data area.
- **Evidence locations:** [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/auth_routes.py`](../routes/auth_routes.py); [`routes/api_token_routes.py`](../routes/api_token_routes.py); [`src/secret_storage.py`](../src/secret_storage.py); [`routes/vault_routes.py`](../routes/vault_routes.py); [`src/tools/vault.py`](../src/tools/vault.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py).
- **Known problems:** Vault-command handling and local application secret storage are distinct paths with different storage mechanisms. This is a source-confirmed boundary, not evidence that either path is compromised.
- **Open question:** What are the current confidentiality, ownership, rotation, and backup semantics for vault session data?
## Persistence, background work, and operations
- **How it works:** SQLite models and persistence are centred in [`core/database.py`](../core/database.py); managers use application data paths. The scheduler and background-job monitor can continue work outside a live browser request. Operational routes cover cleanup, backup, and administrative wipe; the repository also provides a backup script and user documentation.
- **Evidence locations:** [`core/database.py`](../core/database.py); [`src/runtime_paths.py`](../src/runtime_paths.py); [`src/task_scheduler.py`](../src/task_scheduler.py); [`src/bg_jobs.py`](../src/bg_jobs.py); [`src/bg_monitor.py`](../src/bg_monitor.py); [`routes/backup_routes.py`](../routes/backup_routes.py); [`routes/cleanup/cleanup_routes.py`](../routes/cleanup/cleanup_routes.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py); [`scripts/odysseus-backup`](../scripts/odysseus-backup); [`docs/backup-restore.md`](../docs/backup-restore.md).
- **Known problems:** None recorded by this mapping.
- **Open question:** What current behaviour applies to background execution, cancellation, retries, and authority inheritance?
+1
View File
@@ -67,6 +67,7 @@ services:
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
+1
View File
@@ -66,6 +66,7 @@ services:
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
+1
View File
@@ -55,6 +55,7 @@ services:
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
-87
View File
@@ -1,87 +0,0 @@
# Architecture
> [!NOTE]
> This document is the proposed canonical destination for stable high-level
> architecture facts. It remains subject to maintainer review. Source code,
> tests, and configuration remain authoritative for implementation-sensitive
> behaviour.
## Purpose
This document identifies the stable runtime boundaries and primary ownership
locations used to navigate and extend Odysseus.
It intentionally excludes generated metrics, file-size rankings, refactor
priorities, unresolved investigation findings, and proposed package layouts.
## Runtime structure
| Area | Responsibility |
|---|---|
| `app.py` | FastAPI application composition and primary application entry point |
| `launcher.py` | Application launch support |
| `setup.py` | Native setup workflow |
| `core/` | Authentication, middleware, persistence, sessions, and platform primitives |
| `routes/` | HTTP and API route handlers |
| `src/` | Application orchestration, tools, providers, and runtime helpers |
| `services/` | Domain-oriented service implementations |
| `mcp_servers/` | Built-in MCP server implementations |
| `scripts/` | CLI tools, diagnostics, maintenance, and migration helpers |
| `static/` | No-build browser frontend and bundled assets |
| `tests/` | Automated tests and supporting test infrastructure |
## Subsystem boundaries
| Subsystem | Primary implementation locations |
|---|---|
| Application startup | `app.py`, `src/app_initializer.py`, `core/` |
| Authentication and sessions | `core/auth.py`, `core/middleware.py`, `core/session_manager.py`, `routes/auth_routes.py` |
| Chat and streaming | `routes/chat_routes.py`, `routes/chat_helpers.py`, `src/chat_handler.py`, `src/chat_processor.py`, `src/llm_core.py` |
| Agents and tools | `src/agent_loop.py`, `src/tool_execution.py`, `src/agent_tools/`, `src/tools/`, `src/tool_policy.py`, `src/tool_security.py` |
| Models and providers | `routes/model_routes.py`, `src/model_discovery.py`, `src/model_capabilities.py`, `src/endpoint_resolver.py`, `src/llm_core.py` |
| Cookbook and hardware fit | `routes/cookbook_routes.py`, `routes/cookbook_helpers.py`, `src/cookbook_serve_lifecycle.py`, `services/hwfit/` |
| Search and research | `routes/search_routes.py`, `services/search/`, `routes/research/`, `services/research/`, `src/deep_research.py` |
| Documents and retrieval | `routes/document_routes.py`, `src/document_processor.py`, `src/personal_docs.py`, `src/rag_manager.py`, `src/pdf_runtime.py` |
| Memory and skills | `routes/memory/`, `services/memory/`, `routes/skills_routes.py` |
| Email | `routes/email_routes.py`, `routes/email_helpers.py`, `routes/email_pollers.py`, `mcp_servers/email_server.py` |
| Calendar, contacts, notes, and tasks | `routes/calendar_routes.py`, `routes/contacts/`, `routes/note/`, `routes/task_routes.py`, `src/task_scheduler.py` |
| Media and speech | `routes/gallery/`, `routes/stt_routes.py`, `routes/tts_routes.py`, `services/stt/`, `services/tts/` |
| Persistence and operations | `core/database.py`, `src/runtime_paths.py`, `src/bg_jobs.py`, `routes/backup_routes.py`, `routes/cleanup/` |
## Architectural constraints
- Preserve established import and compatibility paths unless a focused change
explicitly migrates them.
- Keep HTTP concerns in route modules and reusable domain behaviour in runtime
or service modules.
- Treat shared persistence, agent orchestration, tool execution, and application
startup as high-authority boundaries.
- Change one ownership boundary at a time.
- Do not mix structural movement with unrelated feature behaviour.
- Validate affected imports, startup paths, compatibility surfaces, and tests.
## Frontend
The browser frontend is a no-build ES-module application under `static/`.
Its maintained module-level structure is documented in
[`static/js/MODULE_SUMMARY.md`](../static/js/MODULE_SUMMARY.md).
## Investigation and snapshots
Non-canonical investigation material is maintained under [`discovery/`](../discovery/).
The following documents may contain dated observations, metrics, unresolved
questions, or historical planning and must not be treated as specifications:
- [`discovery/system-map.md`](../discovery/system-map.md)
- [`discovery/architecture-runtime-inventory.md`](../discovery/architecture-runtime-inventory.md)
## Documentation authority
- Code, tests, and configuration define implemented behaviour.
- Mature specifications define accepted subsystem behaviour where they exist.
- Following maintainer acceptance, this document will define the high-level
architecture map.
- Discovery documents preserve evidence and uncertainty but remain
non-canonical.
-72
View File
@@ -1,72 +0,0 @@
# Documentation style
This guide defines the shared structure and writing conventions for Odysseus documentation.
## Principles
- Write for a clear audience and purpose.
- State whether a document is canonical, informational, a snapshot, or planning material.
- Prefer current behaviour over historical explanation.
- Link to source files, tests, issues, or other documentation when useful.
- Separate verified behaviour from assumptions, open questions, and future work.
- Keep headings descriptive and consistent.
- Use Markdown callouts where status or risk must be visible.
- Do not use emojis.
## Document status
Use a status callout near the top when the document is not normal canonical guidance.
### Canonical documentation
> [!IMPORTANT]
> This document describes accepted current behaviour. Verify implementation-sensitive details against the current code and tests.
### Discovery material
> [!NOTE]
> This is non-canonical discovery material. It records code-grounded observations and open questions.
### Snapshot or inventory
> [!WARNING]
> This document is a dated snapshot. Counts, paths, and implementation details may drift as the codebase changes.
### Planning material
> [!NOTE]
> This document records planning context. It does not define current runtime behaviour or guarantee future implementation.
## Recommended structure
Use the following sections where relevant:
1. Title
2. Purpose or status callout
3. Scope
4. Current behaviour or guidance
5. Safety, limitations, or known gaps
6. Validation or evidence
7. Related documentation
Not every document needs every section.
## Writing style
- Use concise sentences.
- Prefer direct language.
- Avoid jokes, filler, and informal warnings.
- Avoid repeating the same guidance across several files.
- Link to the owning document instead of duplicating large sections.
- Use lists for procedures, requirements, and comparisons.
- Use tables only when they improve scanning.
- Use fenced code blocks with an appropriate language identifier.
- Use relative repository links for internal files.
## Authority
The current code, tests, and configuration are the source of truth for implemented behaviour.
Canonical documentation describes accepted behaviour and supported workflows.
Discovery, inventory, and planning documents must identify themselves explicitly and must not silently become behavioural specifications.
+2 -34
View File
@@ -1,13 +1,7 @@
# Attachment References and Upload Storage
> [!NOTE]
> This document records the current attachment-reference and upload-lifecycle
> contract proposed for maintainer acceptance. Source code, tests, and
> configuration remain authoritative for implementation-sensitive behaviour.
Odysseus stores chat and document attachment bytes under the configured upload
directory and passes stable references through chat history, document flows, and
tool context.
Odysseus stores uploaded bytes once under the configured upload directory and
passes stable references through chat history, tools, and future artifact work.
The goal is to avoid duplicating large inline media payloads in
`chat_messages.content` or the SQLite FTS index.
@@ -60,32 +54,6 @@ External MCP/custom tools should treat the URI and attachment ID as the stable
contract and request bytes through an owner-checked server path, not by assuming
host filesystem layout.
## Implementation evidence
The current contract is implemented primarily through:
- `src/upload_handler.py` for upload metadata, owner-aware resolution,
reservations, cleanup, and deletion;
- `src/attachment_refs.py` for compact persisted references and search-index
sanitization;
- `src/document_processor.py` for resolving attachments into chat/model context;
- `src/tool_execution.py` for attachment manifests exposed to tools;
- `routes/upload_routes.py` and `routes/document_helpers.py` for upload and
retrieval paths.
Focused regression coverage includes:
- `tests/test_attachment_refs.py`;
- `tests/test_upload_handler_cleanup.py`;
- `tests/test_replace_messages_upload_reservations.py`;
- `tests/test_resolve_upload_path_nondict.py`;
- `tests/test_chat_preprocess_tool_policy.py`;
- the upload, attachment, and PDF-marker cases in
`tests/test_security_regressions.py`.
These tests cover compact persistence, owner isolation, path containment,
cleanup safety, reservation-before-write behaviour, and traversal resistance.
## Retention and Deletion
Current retention behavior is conservative:
+2 -2
View File
@@ -181,8 +181,8 @@ Dirty, blocked, conflicting, and unknown merge states are shown as risk/caution
## Validation
```bash
venv/bin/python -m py_compile scripts/pr_blocker_audit.py tests/test_pr_blocker_audit.py
venv/bin/python -m pytest tests/test_pr_blocker_audit.py -q
python3 -m py_compile scripts/pr_blocker_audit.py tests/test_pr_blocker_audit.py
python3 -m pytest tests/test_pr_blocker_audit.py -q
python3 scripts/pr_blocker_audit.py --help
git diff --check
```
+16 -35
View File
@@ -1,28 +1,6 @@
# Odysseus Setup Guide
This guide covers installation, deployment, troubleshooting, and configuration.
For a minimal Docker installation, start with the
[repository README](../README.md#quick-start).
## On this page
- [Quick Start](#quick-start)
- [Docker](#docker-recommended)
- [Native Linux and macOS](#native-linux--macos)
- [Apple Silicon](#apple-silicon)
- [Native Windows](#native-windows)
- [Troubleshooting and advanced setup](#troubleshooting--advanced-setup)
- [Security notes](#security-notes)
- [Configuration](#configuration)
- [Architecture](#architecture)
- [Data and backups](#data)
Related guidance:
- [Security policy](../SECURITY.md)
- [Architecture overview](ARCHITECTURE.md)
- [Backup and restore guide](backup-restore.md)
- [Contributing guide](../CONTRIBUTING.md)
This page keeps the detailed install, deployment, troubleshooting, and configuration notes out of the front README.
## Quick Start
@@ -37,8 +15,8 @@ On first setup, Odysseus creates an admin account (`admin` unless
For Docker installs, the same line is in `docker compose logs odysseus`.
Use that for the first login, then change it in **Settings**.
Contributing? See the [contributing guide](../CONTRIBUTING.md) for development
setup, testing, and pull request guidelines.
Contributing? See [CONTRIBUTING.md](../CONTRIBUTING.md) for setup, testing, and
pull request guidelines.
### Docker (recommended)
```bash
@@ -463,8 +441,8 @@ uv pip sync requirements.lock # reproduce it exactly la
### Outlook / Office 365 email
Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
passwords will fail. See the [Outlook and Microsoft 365 email guide](email-outlook.md)
for the current limitation and planned integration direction.
passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
current limitation and the planned integration direction.
## Security Notes
Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
@@ -552,16 +530,19 @@ npx -y @playwright/mcp@latest --version
That installs `@playwright/mcp` plus Playwright (~300MB total). Restart Odysseus and the server will register at startup.
## Architecture
For stable high-level runtime structure, subsystem boundaries, and documentation
authority, see the [architecture overview](ARCHITECTURE.md).
Source code, tests, and configuration remain authoritative for
implementation-sensitive behaviour.
```
app.py # FastAPI entry point
core/ auth, database, middleware, constants
src/ llm_core, agent_loop, agent_tools, chat_processor, search/
routes/ chat, session, document, memory, model … endpoints
services/ docs, memory, search, hwfit (Cookbook) …
static/ index.html + app.js + style.css + js/ (modular front-end)
docs/ landing page (index.html) + preview clips
```
## Data
All user data lives in `data/` (gitignored): `app.db` (sessions, messages, documents),
`memory.json`, `presets.json`, `uploads/`, `personal_docs/`, `chroma/`, `settings.json`.
To protect or recover this data, follow the
[backup and restore guide](backup-restore.md).
To back up or restore everything in `data/`, see the
[Backup & Restore guide](backup-restore.md).
+1 -1
View File
@@ -33,4 +33,4 @@ PyMuPDF
# magika (onnxruntime), already a core dep via fastembed. We avoid the
# [all]/Azure/audio extras (cloud + heavy). Pinned to a release >30 days old per
# the dependency-age discussion in issue #485.
markitdown[docx,pptx,xlsx,xls]==0.1.6
markitdown[docx,pptx,xlsx,xls]==0.1.7
+6 -3
View File
@@ -3,9 +3,9 @@ uvicorn
python-multipart
python-dotenv
httpx
httpcore>=1.0,<2.0
httpcore>=1.0.9,<2.0
pydantic>=2.13.4
pydantic-settings>=2.14.1
pydantic-settings>=2.14.2
SQLAlchemy
pypdf
beautifulsoup4
@@ -38,7 +38,10 @@ python-dateutil
caldav
cryptography
bcrypt
mcp
# Built-in servers use the v1 low-level Server decorator API. MCP SDK v2 is a
# breaking rewrite, so keep fresh installs on the maintained v1 line until the
# servers are migrated together.
mcp<3
pyotp
qrcode[pil]
croniter
+5
View File
@@ -0,0 +1,5 @@
"""Search route domain package (slice 2j, #4082/#4071).
Contains search_routes.py, migrated from the flat routes/ directory.
Backward-compat shim at routes/search_routes.py re-exports from here.
"""
+111
View File
@@ -0,0 +1,111 @@
"""Search routes — /api/search/config GET, /api/search POST."""
import logging
from typing import Dict, Any
from fastapi import APIRouter, Request
import time
from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO
from services.search.core import _call_provider
from services.search.providers import _get_provider_key, _get_search_instance
logger = logging.getLogger(__name__)
async def _request_values(request: Request) -> Dict[str, Any]:
"""Accept JSON, form data, or query params for search endpoints.
The browser UI posts FormData, while the agent's generic app_api tool
posts JSON. FastAPI Form(...) rejects JSON with a 422 before our handler
runs, which made the model think SearXNG was broken.
"""
values: Dict[str, Any] = dict(request.query_params)
content_type = (request.headers.get("content-type") or "").lower()
try:
if "application/json" in content_type:
body = await request.json()
if isinstance(body, dict):
values.update(body)
else:
form = await request.form()
values.update(dict(form))
except Exception:
pass
return values
def setup_search_routes(config) -> APIRouter:
router = APIRouter(tags=["search"])
@router.get("/api/search/config")
async def get_search_settings() -> Dict[str, Any]:
return get_search_config()
@router.post("/api/search")
async def do_web_search(request: Request) -> Dict[str, Any]:
"""Standalone web search — returns context string + source list.
Used by Compare mode to pre-search once and share results across panes.
"""
values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip()
if not query:
return {"context": "", "sources": [], "error": "query is required"}
time_filter = values.get("time_filter") or values.get("freshness")
if time_filter is not None:
time_filter = str(time_filter).strip() or None
try:
context, sources = comprehensive_web_search(
query, return_sources=True, time_filter=time_filter,
)
return {"context": context, "sources": sources}
except Exception as e:
logger.error(f"Standalone web search failed: {e}")
return {"context": "", "sources": [], "error": str(e)}
@router.get("/api/search/providers")
async def list_search_providers():
"""Return available search providers with config status."""
providers = []
for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items():
if pid == "disabled":
continue
available = True
if needs_key and not _get_provider_key(pid):
available = False
if needs_url and pid == "searxng" and not _get_search_instance():
available = False
providers.append({
"id": pid,
"label": label,
"available": available,
})
return providers
@router.post("/api/search/query")
async def search_with_provider(request: Request) -> Dict[str, Any]:
"""Search using a specific provider. Used by compare search mode."""
values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip()
provider = str(values.get("provider") or "").strip()
try:
count = int(values.get("count") or values.get("limit") or 10)
except Exception:
count = 10
if not query:
return {"results": [], "provider": provider, "error": "query is required"}
if provider not in PROVIDER_INFO or provider == "disabled":
return {"results": [], "provider": provider, "error": "Unknown provider"}
t0 = time.time()
try:
results = _call_provider(provider, query, min(count, 20))
elapsed = round(time.time() - t0, 2)
return {"results": results, "provider": provider, "time": elapsed}
except Exception as e:
elapsed = round(time.time() - t0, 2)
logger.error(f"Search provider {provider} failed: {e}")
return {"results": [], "provider": provider, "time": elapsed, "error": str(e)}
return router
+9 -107
View File
@@ -1,111 +1,13 @@
"""Search routes — /api/search/config GET, /api/search POST."""
"""Backward-compat shim — canonical location is routes/search/search_routes.py.
import logging
from typing import Dict, Any
This module is replaced in ``sys.modules`` by the canonical module object so
that ``import routes.search_routes`` and ``from routes.search_routes import X``
keep resolving to the canonical module. Keeps existing import paths working
after slice 2j (#4082/#4071).
"""
from fastapi import APIRouter, Request
import sys as _sys
import time
from routes.search import search_routes as _canonical # noqa: F401
from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO
from services.search.core import _call_provider
from services.search.providers import _get_provider_key, _get_search_instance
logger = logging.getLogger(__name__)
async def _request_values(request: Request) -> Dict[str, Any]:
"""Accept JSON, form data, or query params for search endpoints.
The browser UI posts FormData, while the agent's generic app_api tool
posts JSON. FastAPI Form(...) rejects JSON with a 422 before our handler
runs, which made the model think SearXNG was broken.
"""
values: Dict[str, Any] = dict(request.query_params)
content_type = (request.headers.get("content-type") or "").lower()
try:
if "application/json" in content_type:
body = await request.json()
if isinstance(body, dict):
values.update(body)
else:
form = await request.form()
values.update(dict(form))
except Exception:
pass
return values
def setup_search_routes(config) -> APIRouter:
router = APIRouter(tags=["search"])
@router.get("/api/search/config")
async def get_search_settings() -> Dict[str, Any]:
return get_search_config()
@router.post("/api/search")
async def do_web_search(request: Request) -> Dict[str, Any]:
"""Standalone web search — returns context string + source list.
Used by Compare mode to pre-search once and share results across panes.
"""
values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip()
if not query:
return {"context": "", "sources": [], "error": "query is required"}
time_filter = values.get("time_filter") or values.get("freshness")
if time_filter is not None:
time_filter = str(time_filter).strip() or None
try:
context, sources = comprehensive_web_search(
query, return_sources=True, time_filter=time_filter,
)
return {"context": context, "sources": sources}
except Exception as e:
logger.error(f"Standalone web search failed: {e}")
return {"context": "", "sources": [], "error": str(e)}
@router.get("/api/search/providers")
async def list_search_providers():
"""Return available search providers with config status."""
providers = []
for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items():
if pid == "disabled":
continue
available = True
if needs_key and not _get_provider_key(pid):
available = False
if needs_url and pid == "searxng" and not _get_search_instance():
available = False
providers.append({
"id": pid,
"label": label,
"available": available,
})
return providers
@router.post("/api/search/query")
async def search_with_provider(request: Request) -> Dict[str, Any]:
"""Search using a specific provider. Used by compare search mode."""
values = await _request_values(request)
query = str(values.get("query") or values.get("q") or "").strip()
provider = str(values.get("provider") or "").strip()
try:
count = int(values.get("count") or values.get("limit") or 10)
except Exception:
count = 10
if not query:
return {"results": [], "provider": provider, "error": "query is required"}
if provider not in PROVIDER_INFO or provider == "disabled":
return {"results": [], "provider": provider, "error": "Unknown provider"}
t0 = time.time()
try:
results = _call_provider(provider, query, min(count, 20))
elapsed = round(time.time() - t0, 2)
return {"results": results, "provider": provider, "time": elapsed}
except Exception as e:
elapsed = round(time.time() - t0, 2)
logger.error(f"Search provider {provider} failed: {e}")
return {"results": [], "provider": provider, "time": elapsed, "error": str(e)}
return router
_sys.modules[__name__] = _canonical
+1 -1
View File
@@ -1409,7 +1409,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
# Prefer the configured DEFAULT (→ Utility) model — not the current chat
# session's model. Fall back to the caller's session model only if unset.
url, model, headers = resolve_endpoint("default", owner=user)
url, model, headers = resolve_endpoint("utility", owner=user)
if not url or not model:
url = url or ((body.get("endpoint_url") or "").strip() or None)
model = model or ((body.get("model") or "").strip() or None)
+2 -2
View File
@@ -50,7 +50,7 @@ import json
import logging
import re
from dataclasses import dataclass, field
from datetime import datetime
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
@@ -441,4 +441,4 @@ class Skill:
def _now_iso() -> str:
return datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+53
View File
@@ -2,6 +2,7 @@
"""Multi-provider TTS service — dispatches to local Kokoro, OpenAI-compatible API, or browser."""
import io
import os
import wave
import logging
import hashlib
@@ -41,6 +42,11 @@ class TTSService:
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
self._kokoro = None # lazy-init
try:
self.max_cache_bytes = int(os.getenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", 500 * 1024 * 1024))
except ValueError:
self.max_cache_bytes = 500 * 1024 * 1024
# ── Settings ──
@@ -89,6 +95,53 @@ class TTSService:
ext = ".mp3" if (len(data) >= 3 and (data[:3] == b'ID3' or (data[0] == 0xff and (data[1] & 0xe0) == 0xe0))) else ".wav"
(self.cache_dir / f"{key}{ext}").write_bytes(data)
self._enforce_cache_limit()
def _enforce_cache_limit(self):
"""Evicts oldest files if the cache exceeds the configured byte limit."""
if self.max_cache_bytes <= 0:
return
try:
files = []
total_size = 0
# Safely scan files and sum sizes, ignoring files deleted mid-scan
for f in self.cache_dir.iterdir():
try:
if f.is_file() and f.suffix.lower() in (".mp3", ".wav"):
files.append(f)
total_size += f.stat().st_size
except OSError:
continue
if total_size > self.max_cache_bytes:
logger.info(
f"TTS cache ({total_size} bytes) exceeded limit ({self.max_cache_bytes} bytes). Evicting oldest files."
)
# Sort files by modification time (oldest first)
try:
files.sort(key=lambda f: f.stat().st_mtime)
except OSError as e:
logger.warning(f"Failed to sort cache files by mtime: {e}")
# Trim down to 80% of max capacity
target_size = self.max_cache_bytes * 0.8
while files and total_size > target_size:
f = files.pop(0)
try:
size = f.stat().st_size
f.unlink()
total_size -= size
except OSError as e:
logger.warning(f"Failed to evict cache file {f}: {e}")
continue
except Exception as e:
logger.warning(f"Error enforcing TTS cache limit: {e}", exc_info=True)
def clear_cache(self):
count = 0
for f in self.cache_dir.glob("*.*"):
+406 -25
View File
@@ -1,31 +1,412 @@
# Architecture runtime inventory disposition
# Architecture Runtime Inventory
> [!NOTE]
> The dated architecture runtime inventory has been classified and transferred
> to [`discovery/architecture-runtime-inventory.md`](../discovery/architecture-runtime-inventory.md)
> after verification of its stable facts.
>
> Stable runtime structure and subsystem boundaries were transferred to the
> proposed canonical destination,
> [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md), for maintainer review.
> **Purpose**: Phase 0 planning baseline for codebase readability improvements (#4071).
> **Parent issue**: [#4082](https://github.com/odysseus-dev/odysseus/issues/4082)
> **Last updated**: dev@b58af42 | 2026-06-16
> **Status**: Draft — to be reviewed before follow-up slices open.
> **Snapshot basis**: Importer / file / import-line counts are refreshed to `dev@b58af42` (2026-06-16) and are recomputable via the commands in §3.4. **Line counts** in §2.1 / §2.2 are a snapshot from an earlier baseline and drift as `dev` moves — recompute any of them with `wc -l <file>`. This inventory tracks structure and risk, not live metrics.
## Classification
This document maps the current runtime module structure, identifies high-risk boundaries, and recommends safe first refactor slices. It does **not** move files, change imports, or alter runtime behavior.
| Content | Disposition |
|---|---|
| Stable runtime structure | Proposed canonical architecture document |
| Stable subsystem boundaries | Proposed canonical architecture document |
| Frontend module organization | `static/js/MODULE_SUMMARY.md` |
| Counts, rankings, and generated metrics | Non-canonical discovery inventory |
| Investigation context and unresolved questions | Non-canonical discovery material |
| Refactor options and prioritization | Issues, Plane, or discovery material |
---
No dated metrics, prioritization, or investigation findings remain in this
specification path.
## 1. Current Structure Overview
## Review status
### 1.1 Top-Level Layout
- Findings and scope have been validated.
- Transfer verification has completed.
- The final canonical status of `docs/ARCHITECTURE.md` requires maintainer review.
- No implementation behaviour, test taxonomy, or runtime files were changed.
```
odysseus/
├── app.py # FastAPI app entrypoint (1,145 lines)
├── conf/ # Configuration (config.py, settings.py, settings_scrub.py)
├── src/ # 95 flat .py files + 2 subdirectories
│ ├── agent_tools/ # Tool helpers: document, filesystem, subprocess, web
│ └── search/ # Search subsystem
├── routes/ # 54 flat .py files — HTTP route handlers
├── core/ # 10 files — database models, auth, middleware, session
├── mcp_servers/ # 5 files — MCP server implementations
├── scripts/ # CLI tools and one-shot scripts
├── static/ # Frontend HTML/CSS/JS
├── tests/ # 583 test files (~54,800 lines)
└── services/ # (exists as needed)
```
### 1.2 Directory Flatness Metric
| Directory | Flat `.py` Files | Subdirectories | Concern |
|-----------|-----------------|----------------|---------|
| `src/` | **95** | 2 (`agent_tools/`, `search/`) | No domain grouping; 95 files in one directory |
| `routes/` | **54** | 0 | All route handlers in one flat directory |
| `core/` | 10 | 0 | Manageable, but `database.py` is oversized |
---
## 2. Largest Runtime Modules
### 2.1 Python Backend
| Rank | File | Lines | Classes | Functions | Risk |
|------|------|-------|---------|-----------|------|
| 1 | `src/tool_implementations.py` | **4,032** | 0 | ~48 | **HIGH** |
| 2 | `routes/email_routes.py` | **3,245** | — | — | **MEDIUM** |
| 3 | `routes/cookbook_routes.py` | **2,969** | — | — | **MEDIUM** |
| 4 | `src/agent_loop.py` | **2,961** | 0 | ~24 | **HIGH** |
| 5 | `src/task_scheduler.py` | **2,330** | — | 5 | MEDIUM |
| 6 | `routes/model_routes.py` | **2,266** | — | — | MEDIUM |
| 7 | `core/database.py` | **2,265** | 28 | ~59 helpers | **HIGH** |
| 8 | `src/builtin_actions.py` | **2,262** | 2 | ~24 | MEDIUM |
| 9 | `src/llm_core.py` | **2,164** | — | — | MEDIUM |
| 10 | `mcp_servers/email_server.py` | 2,197 | — | — | LOW (separate process) |
| 11 | `src/visual_report.py` | 1,918 | — | — | LOW |
| 12 | `routes/gallery_routes.py` | 1,896 | — | — | LOW |
| 13 | `src/ai_interaction.py` | 1,846 | — | — | MEDIUM |
| 14 | `routes/document_routes.py` | 1,717 | — | — | LOW |
| 15 | `routes/skills_routes.py` | 1,648 | — | — | LOW |
**Heuristic**: Files > 2,000 lines with 20+ public symbols and many importers are the highest-risk splits. Files 1,0002,000 lines are medium-risk if tightly coupled.
### 2.2 Frontend
| File | Lines | Concern |
|------|-------|---------|
| `static/style.css` | **36,653** | Entire app CSS in one file (tracked separately in #2617) |
| `static/js/document.js` | **9,776** | Single JS file for document functionality |
| `static/js/slashCommands.js` | 6,498 | |
| `static/js/settings.js` | 5,266 | |
| `static/js/emailLibrary.js` | 5,217 | |
| `static/js/notes.js` | 5,124 | |
| `static/js/chat.js` | 4,985 | |
| `static/app.js` | 4,090 | |
**Note**: Frontend modularization is tracked separately in #2617 (CSS) and is not the focus of this Phase 0 inventory. Frontend is listed here for completeness but follow-up slices should target Python backend boundaries first.
---
## 3. Import Dependency Graph
### 3.1 Who Depends on `core/database.py`
**102 files** import from `core.database` — this is the most depended-upon module:
- All route handlers (`routes/*.py`)
- Most `src/*.py` files
- `core/session_manager.py`, `core/auth.py`
- Multiple test files
**Implication**: Any split of `core/database.py` is the highest-risk refactor. It should be tackled **last**, never first.
### 3.2 Who Depends on `src/tool_implementations.py`
**17 files** import from `src.tool_implementations`:
- `src/agent_loop.py`, `src/builtin_actions.py`, `src/tool_index.py`
- `src/task_scheduler.py`, `src/tool_policy.py`
- Various tests
### 3.3 Who Depends on `src/agent_loop.py`
**22 files** import from `src.agent_loop`:
- `src/tool_policy.py`, `src/teacher_escalation.py`, `src/bg_monitor.py`
- `src/task_scheduler.py`
- Multiple test files
### 3.4 Cross-Layer Import Violations
**`src/` importing from `routes/`** (backwards dependency — domain logic depending on HTTP layer):
```
src/tool_implementations.py ──→ routes/calendar_routes.py
src/tool_implementations.py ──→ routes/cookbook_helpers.py
src/tool_implementations.py ──→ routes/email_helpers.py
src/tool_implementations.py ──→ routes/email_pollers.py
src/tool_implementations.py ──→ routes/email_routes.py
src/tool_implementations.py ──→ routes/model_routes.py
src/tool_implementations.py ──→ routes/note_routes.py
src/tool_implementations.py ──→ routes/prefs_routes.py
```
> These are **runtime imports** (inside function bodies, not at module top), which mitigates circular import risk but indicates fuzzy layer boundaries. Function-level inline imports from the HTTP layer into business logic are a code smell.
**Import counts (top-level)**:
| Direction | Count | Notes |
|-----------|-------|-------|
| `routes/``src/` | **374** | Expected: HTTP handlers call domain logic |
| `routes/``core/` | **126** | Expected: handlers access DB models |
| `src/``routes/` | **31** | **Unexpected**: domain logic reaching into HTTP layer (direct grep of import lines referencing `routes/`) |
| `src/``core/` | **106** | Acceptable but could be reduced with a data-access layer |
> **How the metrics in this document are computed** — recompute against current `dev` before treating any count as authoritative (the tree drifts; these numbers are a snapshot, not a live value):
> - `src/` flat `.py` files: `find src -maxdepth 1 -name '*.py' | wc -l`
> - `tests/` test files: `find tests -name 'test_*.py' | wc -l`
> - `core.database` importers: `grep -rlE '(from|import) +core\.database' --include='*.py' . | grep -v core/database.py | wc -l`
> - `src.agent_loop` importers: `grep -rlE '(from|import) +src\.agent_loop' --include='*.py' . | grep -v src/agent_loop.py | wc -l`
> - Cross-layer import lines: `grep -rhE '(from|import) +<pkg>' --include='*.py' <dir>/ | wc -l` (e.g. `(from|import) +routes` over `src/`)
---
## 4. Route Ownership Map
Routes can be grouped into logical feature domains. Current flat structure obscures these boundaries:
| Domain | Route Files | Total Lines | Review Complexity |
|--------|-------------|-------------|-------------------|
| **Email** | `email_routes.py`, `email_helpers.py`, `email_pollers.py` | 5,936 | HIGH — most complex domain |
| **Chat / Agent** | `chat_routes.py`, `chat_helpers.py`, `shell_routes.py`, `codex_routes.py`, `skills_routes.py` | 6,365 | HIGH — core interaction surface |
| **Cookbook** | `cookbook_routes.py`, `cookbook_helpers.py`, `cookbook_output.py` | 4,110 | MEDIUM |
| **Model / LLM** | `model_routes.py`, `assistant_routes.py`, `copilot_routes.py` | 2,764 | MEDIUM |
| **Calendar / Contacts** | `calendar_routes.py`, `contacts_routes.py` | 2,336 | MEDIUM |
| **Documents** | `document_routes.py`, `document_helpers.py` | 1,954 | LOW |
| **Auth** | `auth_routes.py`, `api_token_routes.py`, `device_flow.py` | 1,171 | LOW |
| **Tasks** | `task_routes.py` (standalone) | 1,157 | LOW |
| **Session** | `session_routes.py` (standalone) | 1,287 | LOW |
| **Gallery** | `gallery_routes.py`, `gallery_helpers.py` | 1,896 | LOW |
| **Memory** | `memory_routes.py` | — | LOW |
| **Research** | `research_routes.py` | — | LOW |
| **MCP** | `mcp_routes.py` | — | LOW |
| **Notes** | `note_routes.py` | — | LOW |
| **Other** | `prefs_routes.py`, `upload_routes.py`, `vault_routes.py`, `webhook_routes.py`, `workspace_routes.py`, `search_routes.py`, `history_routes.py`, `hwfit_routes.py`, `preset_routes.py`, `signature_routes.py`, `backup_routes.py`, `cleanup_routes.py`, `diagnostics_routes.py`, `embedding_routes.py`, `emoji_routes.py`, `font_routes.py`, `stt_routes.py`, `tts_routes.py`, `compare_routes.py`, `personal_routes.py`, `editor_draft_routes.py`, `admin_wipe_routes.py`, `chatgpt_subscription_routes.py` | 2,000+ | LOW individual, HIGH cumulative |
---
## 5. Tool Registry & Implementation Boundaries
### 5.1 Current Tool Architecture
| Component | File | Lines | Role |
|-----------|------|-------|------|
| Tool schemas | `src/tool_schemas.py` | 1,392 | JSON Schema tool definitions (Duck-TypedDict) |
| Tool index | `src/tool_index.py` | 542 | RAG-based tool retrieval from ChromaDB |
| Tool implementations | `src/tool_implementations.py` | 4,032 | 33 `do_*` functions — all tool execution logic |
| Tool security | `src/tool_security.py` | — | Owner-scoped tool blocking |
| Tool policy | `src/tool_policy.py` | — | Guide-only directive, plan-mode disabled tools |
| Tool utils | `src/tool_utils.py` | — | Shared tool helpers |
### 5.2 Tool Implementation Categories
The 33 `do_*` functions in `tool_implementations.py` fall into natural domain groups — the basis for slice 1's split in §6.2:
| Category | `do_*` functions | Count |
|----------|------------------|-------|
| **System / config** | `do_manage_skills`, `do_manage_tasks`, `do_manage_endpoints`, `do_manage_mcp`, `do_manage_webhooks`, `do_manage_tokens`, `do_manage_settings`, `do_api_call`, `do_app_api` | 9 |
| **Cookbook / model serving** | `do_download_model`, `do_serve_model`, `do_list_served_models`, `do_stop_served_model`, `do_tail_serve_output`, `do_list_downloads`, `do_cancel_download`, `do_search_hf_models`, `do_adopt_served_model`, `do_list_cookbook_servers`, `do_list_serve_presets`, `do_serve_preset`, `do_list_cached_models` | 13 |
| **Notes** | `do_manage_notes` | 1 |
| **Calendar** | `do_manage_calendar` | 1 |
| **Search** | `do_search_chats` | 1 |
| **Research** | `do_manage_research`, `do_trigger_research` | 2 |
| **Contacts** | `do_resolve_contact`, `do_manage_contact` | 2 |
| **Vault** | `do_vault_search`, `do_vault_get`, `do_vault_unlock` | 3 |
| **Image** | `do_edit_image` | 1 |
| | **Total** | **33** |
> Low-level tools (filesystem, subprocess, web fetch, document parsing) live in `src/agent_tools/`, **not** in `tool_implementations.py` — out of scope for this split.
---
## 6. Risk Assessment & Candidate Slice Ranking
> **Candidate proposals, not a committed plan.** The rankings, package shapes (e.g. `src/pkg/`, `src/domain/`, `src/infra/`, `src/api/`), split ordering, and route-grouping strategy below are **options for maintainer discussion**. Per #4082/#4071, slice ownership and order are settled by maintainers before any follow-up PR. §1–§3 above are the factual current-state inventory.
### 6.1 Risk Scale
| Level | Criteria |
|-------|----------|
| **LOW** | File has ≤3 importers AND ≤500 lines, OR is a pure refactor with clear boundaries |
| **MEDIUM** | File has 415 importers OR 5001,500 lines |
| **HIGH** | File has 16+ importers OR >2,000 lines, OR has cross-layer import violations |
### 6.2 Ranked Split Candidates
| Priority | Target | Risk | Rationale |
|----------|--------|------|-----------|
| **1** | `src/tool_implementations.py``src/tools/*.py` | **MEDIUM** | 4,032 lines → ~10 files by tool category. Already has natural boundaries. 17 importers, tracked in #3629. Use `__init__.py` shim to keep existing imports working. |
| **2** | `routes/` → domain subdirectories (one domain per PR) | **MEDIUM** | 54 flat files. Done **one domain at a time** (e.g. a standalone PR for the email domain, then chat, …), not a broad reorganization — route modules carry helper imports, registration assumptions, and test import paths. |
| **3** | `src/agent_loop.py``src/agent/loop.py` + submodules | **MEDIUM-HIGH** | 2,961 lines, 24 functions. Can extract prompt building, classification, verification, and runaway detection. Tracked in #3266. |
| **4** | `src/``src/pkg/`, `src/domain/`, `src/infra/`, `src/api/` | **MEDIUM** | Structural reorganization. Split flat `src/` into layered packages. Must come after routes and tools are stable. |
| **5** | `routes/email_*.py` consolidation | **LOW** | Already grouped by filename prefix. Low-risk cleanup within the email domain. |
| **6** | `core/database.py``src/infra/database/models/*.py` | **HIGH** | 28 classes, 102 importers. Highest-risk split. Must be **last** in any sequence. Requires careful import shim strategy. |
| **7** | Frontend CSS modularization | **MEDIUM** | 36,653 lines. Tracked in #2617. Separate timeline from backend work. |
| **8** | Frontend JS modularization | **MEDIUM** | 9,776 lines in `document.js`. Introduce ES modules at minimum. |
### 6.3 Candidate First 3 Behavior-Preserving Slices
**Slice 1: Split `tool_implementations.py`** (Lowest-risk high-impact)
- Create `src/tools/` package with one file per tool category
- Add `src/tools/__init__.py` re-exporting all symbols with current names
- Update 17 importers to use new paths (can be deferred via shim)
- Validation: `python -m pytest tests/ -x -q` + manual smoke test of tool execution
- Reference: #3629
**Slice 2: Group `routes/` by domain** (one domain per PR, not a broad sweep)
Route modules carry helper imports, router registration assumptions, and test import paths, so this must be done **one domain at a time** rather than as a single reorganization PR. Example sequence (each its own PR):
- PR 2a: move the **email** domain (`email_routes.py`, `email_helpers.py`, `email_pollers.py`) → `routes/email/` + shim
- PR 2b: move the **chat/agent** domain → `routes/chat/` + shim
- PR 2c: move the **cookbook** domain → `routes/cookbook/` + shim
- …and so on per domain from §4
Each PR: add `__init__.py` re-exporting old names, update `app.py` router imports, validation `python app.py` starts clean. **No behavior change** — pure file reorganization.
**Slice 3: Extract `agent_loop.py` submodules** (Improve reviewability)
- Move prompt assembly → `src/agent/prompt.py`
- Move request classification → `src/agent/classifier.py`
- Move sub-agent verification → `src/agent/verifier.py`
- Move runaway detection → `src/agent/runaway.py`
- Move context management → `src/agent/context.py`
- Keep `src/agent/loop.py` as the main orchestration module
- Validation: `python -m pytest tests/test_agent_loop.py tests/test_loop_breaker_runaway.py -v`
---
## 7. Safety Guardrails for Follow-Up Work
Per maintainer guidance in #4082 and #4071:
- [ ] **One domain/slice per PR** — never mix multiple reorganizations
- [ ] **No behavior changes** mixed with file moves — pure reorganization only
- [ ] **Keep compatibility shims**`__init__.py` re-exports for all existing import paths
- [ ] **Add or identify focused tests** before risky splits
- [ ] **Do not start with `core/database.py`** or broad route movement unless this inventory shows a safe boundary
- [ ] **Prefer small, reviewable slices** over large restructures
- [ ] **No packaging/runtime/tooling migration** mixed into file moves
- [ ] **No frontend framework migration** inside this stabilization lane
- [ ] **Validate with `python -m compileall`** — every PR must pass CI checks
- [ ] **Validate with `pytest`** — run the full test suite before opening each PR
---
## 8. Validation Commands
Each follow-up PR should be verifiable with these commands before submission:
```bash
# Syntax check — must pass with zero errors
python -m compileall src/ routes/ core/ conf/
# Full test suite — must match baseline pass rate
python -m pytest tests/ -x -q
# Import shim verification — existing import paths must still work
python -c "from src.tool_implementations import do_search_chats; print('OK')"
# App startup smoke test (if backend touched)
timeout 5 python app.py 2>&1 | head -5 || true
```
---
## 9. Open Questions
1. Is `#2538` (specs ground truth) the canonical behavior map baseline, and should this inventory be kept in sync with those specs once merged?
2. Should route grouping follow the domain map proposed here, or is there a different taxonomy preferred by maintainers?
3. For the `tool_implementations.py` split (#3629), is the tool categorization in §5.2 acceptable, or should it follow a different grouping?
4. Should compatibility shims (`__init__.py`) be temporary (removed in a follow-up wave) or permanent?
5. Should an ADR (Architecture Decision Record) document be started to track decisions made during this process?
---
## 10. Future Direction (NOT current state)
The following are **future refactor targets** (candidate directions **pending maintainer agreement**, not committed), recorded here so this inventory does not imply they exist today. None of them are present in the current `dev` tree:
- `main.py` — proposed rename of the `app.py` entrypoint. Today the app boots via `app.py`.
- `src/agent/` — proposed package to hold `agent_loop.py` submodules (prompt/classifier/verifier/runaway/context). Today `agent_loop.py` is a single flat file in `src/`.
- `src/infra/`, `src/domain/`, `src/pkg/`, `src/api/` — proposed layered reorganization of the flat `src/` directory (slice 4 in §6).
These become real only when the corresponding slices land.
---
## Appendix A: File Listing
### `src/` (95 files — 61 shown; run `ls src/*.py` for the full list)
```
agent_loop.py tool_implementations.py tool_schemas.py
tool_index.py tool_security.py tool_policy.py
tool_utils.py builtin_actions.py task_scheduler.py
llm_core.py model_context.py model_discovery.py
session_search.py context_budget.py context_compactor.py
ai_interaction.py action_intents.py agent_runs.py
app_helpers.py app_initializer.py config.py
database.py memory.py memory_provider.py
secret_storage.py prompt_security.py url_security.py
url_safety.py rate_limiter.py cleanup_service.py
readiness.py service_health.py exceptions.py
request_models.py assistant_log.py bg_monitor.py
builtin_mcp.py chat_helpers.py chroma_client.py
document_processor.py embedding_lanes.py deep_research.py
research_handler.py research_utils.py personal_docs.py
rag_manager.py rag_singleton.py topic_analyzer.py
visual_report.py youtube_handler.py pdf_forms.py
pdf_form_doc.py pdf_runtime.py caldav_writeback.py
email_thread_parser.py text_helpers.py user_time.py
teacher_escalation.py cookbook_serve_lifecycle.py
chatgpt_subscription.py mcp_manager.py
```
### `routes/` (54 files)
```
__init__.py _validators.py
auth_routes.py api_token_routes.py device_flow.py
chat_routes.py chat_helpers.py shell_routes.py
codex_routes.py skills_routes.py
email_routes.py email_helpers.py email_pollers.py
cookbook_routes.py cookbook_helpers.py cookbook_output.py
model_routes.py assistant_routes.py copilot_routes.py
calendar_routes.py contacts_routes.py
document_routes.py document_helpers.py
gallery_routes.py gallery_helpers.py
task_routes.py session_routes.py
note_routes.py memory_routes.py research_routes.py
mcp_routes.py search_routes.py history_routes.py
webhook_routes.py workspace_routes.py upload_routes.py
vault_routes.py prefs_routes.py preset_routes.py
signature_routes.py personal_routes.py hwfit_routes.py
backup_routes.py cleanup_routes.py diagnostics_routes.py
embedding_routes.py emoji_routes.py font_routes.py
stt_routes.py tts_routes.py compare_routes.py
editor_draft_routes.py chatgpt_subscription_routes.py admin_wipe_routes.py
```
### `core/` (10 files)
```
__init__.py constants.py database.py models.py
auth.py middleware.py session_manager.py exceptions.py
atomic_io.py platform_compat.py
```
---
## Appendix B: Key Import Relationships
```
core/database.py ←── 102 importers (routes/*, src/*, core/*, tests/*)
├── routes/auth_routes.py
├── routes/email_routes.py
├── src/builtin_actions.py
├── src/task_scheduler.py
├── src/tool_implementations.py (inline)
└── ...97 more
src/tool_implementations.py ←── 17 importers
├── src/agent_loop.py
├── src/builtin_actions.py
├── src/tool_index.py
├── src/task_scheduler.py
├── src/tool_policy.py
└── ...12 more (mostly tests)
src/agent_loop.py ←── 22 importers
├── src/tool_policy.py
├── src/teacher_escalation.py
├── src/bg_monitor.py
├── src/task_scheduler.py
└── 18 more (incl. tests)
```
+19 -7
View File
@@ -1237,15 +1237,27 @@ def _anthropic_rejects_temperature(model: str) -> bool:
return False
# `(?<![a-z])` anchors "opus" to a word boundary so a substring match like
# `oct-opus`/`octopus-4-8` can't be read as Opus (it would otherwise strip
# temperature). Cap the minor at 1-2 digits and forbid a trailing digit so a
# dated id like `claude-opus-4-20250514` (Opus 4.0) parses as major-only (no
# minor match, kept) instead of reading the date `20250514` as a giant minor
# that would falsely test >= 4.7. Dated 4.7+ snapshots (`claude-opus-4-7-
# 20260201`) keep their explicit minor and are still matched.
match = re.search(r"(?<![a-z])opus[-_]?(\d+)[-_.](\d{1,2})(?!\d)", model.lower())
# temperature). Both version components are capped at 1-2 digits and forbid a
# trailing digit, so an 8-digit date can never be read as a version number:
# `claude-opus-4-20250514` (Opus 4.0) parses as major-only rather than reading
# `20250514` as a giant minor, and `claude-3-opus-20240229` (legacy Claude 3
# Opus, date directly after "opus-") fails to match at all rather than reading
# the date as a giant major. Dated 4.7+ snapshots (`claude-opus-4-7-20260201`)
# keep their explicit minor and are still matched.
#
# The minor is optional and a missing minor reads as `.0`, so major-only ids
# like `claude-opus-5` are correctly treated as >= 4.7 (issue #5753). Without
# this, every Opus 5 call kept `temperature` and failed with HTTP 400 — visible
# only on paths that pass a temperature, e.g. scheduled tasks inheriting
# `stream_agent_loop`'s 0.3 default, which returned empty responses.
match = re.search(
r"(?<![a-z])opus[-_]?(\d{1,2})(?!\d)(?:[-_.](\d{1,2})(?!\d))?", model.lower()
)
if not match:
return False
return (int(match.group(1)), int(match.group(2))) >= (4, 7)
major = int(match.group(1))
minor = int(match.group(2)) if match.group(2) else 0
return (major, minor) >= (4, 7)
# Reasoning effort level sent to Mistral thinking-capable models. Mistral's
# API accepts "high", "medium", "low", "none" — see
+13 -7
View File
@@ -758,30 +758,36 @@ export function mdToHtml(src, opts) {
// Remove empty paragraphs
s = s.replace(/<p><\/p>/g, '');
// Every restore below passes a function replacer rather than the block string
// itself. With a string replacement, `String.replace` reads `$&`, `` $` ``,
// `$'` and `$$` in the *replacement* as substitution patterns, so a restored
// block containing them is corrupted: `$&` re-inserts the placeholder, `` $` ``
// and `$'` splice in the surrounding document, and `$$` collapses to `$`. Those
// sequences are ordinary content in fenced code (`perl -pe 's/x/$& y/'`,
// `echo "$$USD"`). A function replacer inserts its return value verbatim.
// CRITICAL: Restore allowed HTML blocks first
allowedHtmlBlocks.forEach((block, index) => {
s = s.replace(`___ALLOWED_HTML_${index}___`, block);
s = s.replace(`___ALLOWED_HTML_${index}___`, () => block);
});
// Restore math blocks
mathBlocks.forEach((block, index) => {
s = s.replace(`___MATH_BLOCK_${index}___`, block);
s = s.replace(`___MATH_BLOCK_${index}___`, () => block);
});
// Restore mermaid diagram blocks
mermaidBlocks.forEach((block, index) => {
s = s.replace(`___MERMAID_BLOCK_${index}___`, block);
s = s.replace(`___MERMAID_BLOCK_${index}___`, () => block);
});
// CRITICAL: Restore code blocks at the end
codeBlocks.forEach((block, index) => {
s = s.replace(`___CODE_BLOCK_${index}___`, block);
s = s.replace(`___CODE_BLOCK_${index}___`, () => block);
});
// Restore inline code spans last, so placeholders carried inside restored
// <a>/allowed-HTML blocks are resolved too. The function replacer keeps the
// escaped code literal — e.g. a shell snippet like `echo $1` is not treated
// as a regex back-reference.
// <a>/allowed-HTML blocks are resolved too.
inlineCodeBlocks.forEach((block, index) => {
s = s.replace(`___INLINE_CODE_${index}___`, () => block);
});
+25 -22
View File
@@ -3031,12 +3031,14 @@ async function initEmailAccountsSettings() {
const body = {
name: el('eaf-name').value.trim() || el('eaf-from').value.trim(),
from_address: el('eaf-from').value.trim(),
display_name: el('eaf-display-name').value.trim(),
imap_host: el('eaf-imap-host').value.trim(),
imap_port: parseInt(el('eaf-imap-port').value) || 993,
imap_user: el('eaf-imap-user').value.trim(),
imap_starttls: el('eaf-imap-starttls').checked,
smtp_host: el('eaf-smtp-host').value.trim(),
smtp_port: parseInt(el('eaf-smtp-port').value) || 587,
smtp_security: el('eaf-smtp-security').value,
smtp_user: el('eaf-imap-user').value.trim(),
};
if (!body.name) { el('eaf-msg').textContent = 'Enter a Name or Email first'; el('eaf-msg').style.color = 'var(--red)'; return; }
@@ -5788,29 +5790,30 @@ export function close() {
window.history.replaceState(null, '', clean);
const success = sp.has('email_oauth_success');
const errMsg = sp.get('email_oauth_error') || '';
// Open settings → integrations after the app has initialised.
function _tryOpen() {
if (window.settingsModule && typeof window.settingsModule.open === 'function') {
window.settingsModule.open('integrations');
// Brief toast-style banner.
const banner = document.createElement('div');
banner.textContent = success
? 'Google account connected — email is ready'
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
Object.assign(banner.style, {
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
});
document.body.appendChild(banner);
setTimeout(() => banner.remove(), 4000);
} else {
setTimeout(_tryOpen, 100);
}
// Open settings → integrations once the document is ready. This module owns
// the open() API, so it does not need to wait for a window-level alias.
function _showResult() {
open('integrations');
// Brief toast-style banner.
const banner = document.createElement('div');
banner.textContent = success
? 'Google account connected — email is ready'
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
Object.assign(banner.style, {
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
});
document.body.appendChild(banner);
setTimeout(() => banner.remove(), 4000);
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', _showResult, { once: true });
} else {
_showResult();
}
_tryOpen();
})();
const settingsModule = { open, close, initIntegrations, initUnifiedIntegrations, syncAdminVisibility, refreshAiModelEndpoints };
+175 -142
View File
@@ -1,169 +1,202 @@
# Test layout inventory
> [!NOTE]
> This document is a completed implementation snapshot for the first low-risk
> test-directory reorganization. It does not propose additional file moves and
> is not the canonical definition of the current test taxonomy.
- **Original inventory:** #3712
- **Parent tracker:** #2523
- **Implemented by:** #3842
- **Status:** completed historical snapshot
- **Durable testing policy:** [`TESTING_STANDARD.md`](./TESTING_STANDARD.md)
- **Current testing mechanics:** [`README.md`](./README.md)
## Disposition
This document is retained as a non-canonical historical record of the completed
CLI test relocation.
Reusable testing rules are maintained in `TESTING_STANDARD.md`; current helper,
taxonomy-runner, and execution mechanics are maintained in `README.md`, the test
suite, and its supporting code.
Counts, residual-file observations, and path listings in this document are
snapshot evidence only. They must not be maintained as durable repository truth.
Any additional test relocation requires a separately approved issue or work item.
# Test Layout Inventory
## Purpose
This document preserves the reasoning, scope, and validation contract for the
first mechanical reorganization of the formerly flat `tests/` directory.
Inventory for the first low-risk split of the flat `tests/` directory
(issue #3712, parent #2523). This document only records *what* should move
first and *why*; it moves nothing. The actual move is a separate, mechanical
PR that relocates the listed files verbatim and changes no test content.
The original change moved 28 tests classified as `area_cli` into `tests/cli/`
without changing their assertions, helpers, taxonomy, or runtime behaviour.
The target layout and category definitions come from
[`TESTING_STANDARD.md`](./TESTING_STANDARD.md); the collection-time markers
come from [`_taxonomy.py`](./_taxonomy.py), which classifies by **filename
tokens only** (paths are ignored, except the `tests/helpers/` rule). A file
keeps its `area_*`/`sub_*` markers when moved into a subdirectory, and
`conftest.py` discovers marker names recursively (`rglob`), so a move does not
disturb marker registration or focused selection.
Current testing guidance and classification rules are maintained in:
## Current low-risk candidate groups
- [`README.md`](./README.md)
- [`TESTING_STANDARD.md`](./TESTING_STANDARD.md)
- [`_taxonomy.py`](./_taxonomy.py)
Groups whose tests need no route/app setup and no real DB/session setup:
The current repository and test suite remain authoritative when this snapshot
and the implementation differ.
1. **CLI / script tests** (`area_cli`, 28 files) - load `scripts/` entry
points via `tests.helpers.cli_loader.load_script`; DB access is stubbed
with `tests.helpers.db_stubs` (`SessionLocal` is a plain stub attribute).
No `TestClient`, no FastAPI app import, no SQLite files.
2. **Helper self-tests** (`area_helpers`) - e.g. `test_helpers_import_state.py`,
`test_db_stubs_helper.py`. Safe but tiny (two files), and they test the
shared helpers from the #3685 audit (merged) that the rest of the suite
depends on; little payoff as a first slice.
3. **Pure unit / parsing tests** (`area_unit`) - `*_nonstring.py`,
`*_nondict.py`, parsing tests. Large and heterogeneous; some touch
provider/session modules, so the boundary is less crisp.
4. **Static checks** - e.g. `test_readme_ascii_fenced.py`,
`test_docs_no_orphan_images.py`. Safe but tiny and `uncategorized` in the
taxonomy, so a move buys little and matches no existing marker.
## Why CLI tests were selected
Not candidates for the first move (per #3712 guidance): security/owner-scope
tests, route/API tests, DB/session-heavy tests, auth/session concurrency
tests, and the taxonomy/runner infrastructure tests that changed recently
(#3491, #3556, #3659, #3711).
The CLI and script tests were selected as the first low-risk group because they
formed a narrow, mechanically identifiable boundary:
## Recommended first move
- script entry points were loaded through shared CLI helpers;
- database access used controlled stubs rather than persistent sessions;
- the tests did not require FastAPI application or route setup;
- the tests did not require a real database;
- taxonomy classification could be derived consistently from filenames;
- absolute helper imports remained valid after relocation.
**CLI / script tests → `tests/cli/`**
This reduced the risk relative to route, security, session, provider, and
heterogeneous unit-test groups.
Why this group over the alternatives:
## Completed implementation
- Lowest coupling: every file imports only the script under test (via
`cli_loader`) plus `tests.helpers` stubs - no app, no routes, no real DB.
- Crisp, machine-checkable boundary: the set is exactly the files classified
`area_cli` by `_taxonomy.py`, so before/after selection counts can be
compared mechanically.
- Already the planned target dir for this category in `TESTING_STANDARD.md`
(`tests/cli/`).
- Absolute imports (`from tests.helpers...`) and unique basenames mean no
import-order or module-name collisions after the move.
- Lower risk than helper self-tests (tiny group, little payoff), unit tests
(fuzzy boundary), or anything security/route/session-shaped.
Repository history records the implementation as:
## Files included in the first move
- `83af3ca test: move area_cli tests into cli directory (#3842)`
The 28 files classified `area_cli` (verified against `_taxonomy.py`):
The following 28 files were included in that completed move:
Note: this inventory was refreshed against current `dev` after `tests/test_research_cli_status.py` was added to the `area_cli` set.
- `tests/cli/test_calendar_cli_name.py`
- `tests/cli/test_contacts_cli_rows.py`
- `tests/cli/test_cookbook_cli_state.py`
- `tests/cli/test_docs_cli_content_length.py`
- `tests/cli/test_gallery_cli_album_count.py`
- `tests/cli/test_gallery_cli_preview.py`
- `tests/cli/test_logs_cli_resolve_nonstring.py`
- `tests/cli/test_mail_cli_read_empty_fetch.py`
- `tests/cli/test_mail_cli_recipients.py`
- `tests/cli/test_mcp_cli_env_serialize.py`
- `tests/cli/test_mcp_cli_json.py`
- `tests/cli/test_memory_cli_rows.py`
- `tests/cli/test_notes_cli_items.py`
- `tests/cli/test_personal_cli_rows.py`
- `tests/cli/test_preset_cli_invalid_entries.py`
- `tests/cli/test_preset_cli_set_corrupt_entry.py`
- `tests/cli/test_preset_cli_store.py`
- `tests/cli/test_research_cli_preview.py`
- `tests/cli/test_research_cli_status.py`
- `tests/cli/test_research_cli_status_filter.py`
- `tests/cli/test_research_cli_store.py`
- `tests/cli/test_sessions_cli.py`
- `tests/cli/test_signature_cli_export.py`
- `tests/cli/test_skills_cli_preview.py`
- `tests/cli/test_skills_cli_rows.py`
- `tests/cli/test_tasks_cli_preview.py`
- `tests/cli/test_theme_cli_store.py`
- `tests/cli/test_webhook_cli_mask.py`
- `tests/test_calendar_cli_name.py`
- `tests/test_contacts_cli_rows.py`
- `tests/test_cookbook_cli_state.py`
- `tests/test_docs_cli_content_length.py`
- `tests/test_gallery_cli_album_count.py`
- `tests/test_gallery_cli_preview.py`
- `tests/test_logs_cli_resolve_nonstring.py`
- `tests/test_mail_cli_read_empty_fetch.py`
- `tests/test_mail_cli_recipients.py`
- `tests/test_mcp_cli_env_serialize.py`
- `tests/test_mcp_cli_json.py`
- `tests/test_memory_cli_rows.py`
- `tests/test_notes_cli_items.py`
- `tests/test_personal_cli_rows.py`
- `tests/test_preset_cli_invalid_entries.py`
- `tests/test_preset_cli_set_corrupt_entry.py`
- `tests/test_preset_cli_store.py`
- `tests/test_research_cli_preview.py`
- `tests/test_research_cli_status_filter.py`
- `tests/test_research_cli_status.py`
- `tests/test_research_cli_store.py`
- `tests/test_sessions_cli.py`
- `tests/test_signature_cli_export.py`
- `tests/test_skills_cli_preview.py`
- `tests/test_skills_cli_rows.py`
- `tests/test_tasks_cli_preview.py`
- `tests/test_theme_cli_store.py`
- `tests/test_webhook_cli_mask.py`
## Original exclusions
## Files intentionally excluded
The original inventory deliberately excluded:
- `tests/test_backup_cli_security.py` - classifies as `area_security`
(security outranks cli in the taxonomy); moving it into `tests/cli/` would
make the directory disagree with its marker. It belongs with the security
group in a later phase.
- `tests/test_run_focus.py`, `tests/test_taxonomy.py` - taxonomy/runner
infrastructure tests, recently changed (#3556, #3659); they also pin
flat-layout paths (e.g. `tests/test_auth_config_lock_concurrency.py` in
`test_run_focus.py`), so they stay put.
- Script-like but `uncategorized` files - `test_pr_blocker_audit.py`,
`test_update_database_script.py`, `test_windows_update_script.py`,
`test_setup_admin_user.py`, `test_amd_gpu_check_args.py`, `test_hwfit_*.py`.
They exercise `scripts/` too, but moving them would make `tests/cli/`
diverge from the `area_cli` marker set. Reclassify or move them in a later,
separate slice.
- Everything else (security, routes, services, unit, js, helpers) - out of
scope for the first move by design.
- `tests/test_backup_cli_security.py`, because its security classification
outranked the CLI token in its filename;
- taxonomy and focused-runner infrastructure, including
`tests/test_taxonomy.py` and `tests/test_run_focus.py`;
- script-oriented tests not classified as `area_cli`;
- route, service, security, session, JavaScript, helper, and broad unit-test
groups.
## How this was verified
These exclusions kept the implementation mechanical and prevented it from
mixing taxonomy changes, assertion changes, helper extraction, or unrelated
test-suite restructuring.
## Current repository observation
The current taxonomy classifies 30 files as `area_cli`:
| Location | Files |
|---|---:|
| `tests/cli/` | 28 |
| Flat `tests/` directory | 2 |
The two currently classified CLI tests outside `tests/cli/` are:
- `tests/test_calendar_cli_overlap.py`
- `tests/test_memory_cli_add_nondict.py`
These files were not part of the original 28-file implementation.
Their presence is recorded solely to keep this snapshot accurate. This
documentation update does not recommend moving them, changing their taxonomy,
or opening implementation work. Any future test-layout change requires separate
maintainer agreement, scope, and validation.
## Validation contract preserved by the original move
The completed implementation was designed to preserve:
- identical `area_cli` selection before and after relocation;
- successful direct execution of the relocated tests;
- successful whole-suite collection;
- unchanged taxonomy and focused-runner behaviour;
- no stale references to the former flat paths;
- no production-code changes.
Relevant current verification commands include:
Read-only checks, run from the repo root on this branch. Note the real API is
`classify_test_path` (there is no `classify_test_file`).
```bash
venv/bin/python tests/run_focus.py --dry-run --area cli
venv/bin/python -m pytest -m area_cli -q
venv/bin/python -m pytest tests/cli/ -q
venv/bin/python -m pytest tests/test_taxonomy.py tests/test_run_focus.py -q
venv/bin/python -m pytest --collect-only -q
# Compute the area_cli set and confirm test_backup_cli_security.py is
# area_security. Expected: 28 files, then "security".
./venv/bin/python - <<'PY'
from pathlib import Path
from tests._taxonomy import classify_test_path
cli = [p for p in sorted(Path("tests").glob("test_*.py"))
if classify_test_path(p).area == "cli"]
print(len(cli))
for p in cli:
print(p)
print(classify_test_path("tests/test_backup_cli_security.py").area)
PY
# Coupling check across the CLI files. Expected: the only hits are
# "SessionLocal" as stub attribute names passed to tests.helpers.db_stubs;
# no TestClient, FastAPI, create_app, sqlite, or dependency_overrides.
rg -n "TestClient|FastAPI|create_app|SessionLocal|sqlite|dependency_overrides" \
tests/test_*cli*.py tests/test_sessions_cli.py
# Hard-coded flat paths to the exact CLI files outside tests/. Expected: no matches.
./venv/bin/python - <<'PY2' > /tmp/area_cli_paths.txt
from pathlib import Path
from tests._taxonomy import classify_test_path
for path in sorted(Path("tests").glob("test_*.py")):
if classify_test_path(path).area == "cli":
print(path)
PY2
rg -n -F -f /tmp/area_cli_paths.txt .github scripts docs \
tests/README.md tests/TESTING_STANDARD.md pyproject.toml 2>/dev/null || true
```
These commands describe validation coverage. This snapshot does not claim that
they were executed as part of the present documentation update.
Also checked by reading the code: `tests/conftest.py` registers sub-markers
from a recursive `rglob` scan, and `tests/_taxonomy.py` classifies by filename
tokens only (plus the `tests/helpers/` directory rule), so the markers of the
28 files do not change when they move into `tests/cli/`.
## Validation for the future move PR
Run with the project venv (`./venv/bin/python`); system `python3` may miss
pinned deps. Before the move, record the baseline; after, compare:
```bash
# Selection must match the 28 files before and after the move.
./venv/bin/python tests/run_focus.py --dry-run --area cli
./venv/bin/python -m pytest -m area_cli -q
# Moved files pass when targeted directly.
./venv/bin/python -m pytest tests/cli/ -q
# Whole-suite collection still succeeds (catches import/path breakage).
./venv/bin/python -m pytest --collect-only -q
# Taxonomy/runner infrastructure is unaffected.
./venv/bin/python -m pytest tests/test_taxonomy.py tests/test_run_focus.py -q
# No stale flat-path references to the moved files. Expected: no matches
# outside tests/cli/ itself.
./venv/bin/python - <<'PY2' > /tmp/area_cli_paths.txt
from pathlib import Path
from tests._taxonomy import classify_test_path
for path in sorted(Path("tests").glob("test_*.py")):
if classify_test_path(path).area == "cli":
print(path)
PY2
rg -n -F -f /tmp/area_cli_paths.txt .github scripts docs \
tests/README.md tests/TESTING_STANDARD.md pyproject.toml 2>/dev/null || true
```
Pass criteria: identical test counts for `-m area_cli` before/after, zero
collection errors, and no changes outside the moved files.
## Non-goals
This document does not:
- propose moving or renaming tests;
- redefine taxonomy classifications;
- change test behaviour or assertions;
- prescribe the next test-suite refactor;
- replace the testing standard;
- create implementation work from the two current residual files.
- No file moves, renames, or deletions in this PR.
- No changes to `conftest.py`, `_taxonomy.py`, `run_focus.py`, helpers,
markers, CI workflows, or production code.
- No recommendation to split the whole suite at once; later groups get their
own inventory-then-move slices.
+148 -181
View File
@@ -1,13 +1,5 @@
# Oversized Test File Split Plan
> [!NOTE]
> This is generated planning material based on the current test tree.
>
> It is non-canonical and disposable. Durable testing policy belongs in
> `tests/TESTING_STANDARD.md`; active split selection and prioritization
> belong in approved issues or the project tracker.
> It does not define runtime behaviour or approve any test-file split.
## Purpose
This document plans future oversized test-file splits using current repo data.
@@ -34,8 +26,8 @@ Future split PRs must still inspect each file manually before editing.
## Current summary
- test files scanned: 746
- collected pytest items counted: 4764
- test files scanned: 583
- collected pytest items counted: 3586
- large-file threshold: 300 lines
- large-collected threshold: 20 collected items
@@ -43,123 +35,123 @@ Area distribution:
| Value | Files |
|---|---:|
| cli | 30 |
| cli | 28 |
| helpers | 1 |
| js | 49 |
| routes | 35 |
| security | 88 |
| services | 197 |
| uncategorized | 306 |
| unit | 40 |
| js | 39 |
| routes | 23 |
| security | 77 |
| services | 144 |
| uncategorized | 234 |
| unit | 37 |
Sub-area distribution:
| Value | Files |
|---|---:|
| api | 8 |
| api | 6 |
| atomic | 3 |
| auth | 11 |
| calendar | 12 |
| cli | 30 |
| confinement | 8 |
| cookbook | 20 |
| document | 13 |
| email | 23 |
| embedding | 6 |
| gallery | 6 |
| auth | 9 |
| calendar | 10 |
| cli | 28 |
| confinement | 7 |
| cookbook | 13 |
| document | 11 |
| email | 12 |
| embedding | 3 |
| gallery | 5 |
| history | 3 |
| js | 49 |
| llm | 21 |
| mcp | 12 |
| memory | 17 |
| js | 39 |
| llm | 16 |
| mcp | 8 |
| memory | 15 |
| nondict | 7 |
| nonstring | 23 |
| owner | 15 |
| owner_scope | 27 |
| parse | 5 |
| provider | 16 |
| research | 17 |
| nonstring | 22 |
| owner | 14 |
| owner_scope | 23 |
| parse | 4 |
| provider | 6 |
| research | 16 |
| route | 6 |
| routes | 19 |
| scheduler | 4 |
| routes | 9 |
| scheduler | 3 |
| scope | 5 |
| security | 9 |
| session | 18 |
| ssrf | 6 |
| webhook | 4 |
| session | 16 |
| ssrf | 3 |
| webhook | 3 |
| xss | 5 |
Values below 2 files: 318 values covering 318 files.
Values below 2 files: 244 values covering 244 files.
## Top files by collected pytest items
| File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
|---|---:|---:|---:|---:|---|---|---|
| `tests/test_model_routes.py` | 2099 | 163 | 134 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
| `tests/test_security_regressions.py` | 1515 | 98 | 74 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
| `tests/test_cookbook_helpers.py` | 958 | 69 | 69 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
| `tests/test_shell_routes.py` | 538 | 69 | 53 | 9 | routes | routes | route/api, import-state, filesystem |
| `tests/test_endpoint_probing.py` | 497 | 61 | 36 | 7 | uncategorized | endpoint_probing | route/api, db/session, import-state |
| `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
| `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
| `tests/test_provider_classification.py` | 188 | 67 | 21 | 4 | services | provider | - |
| `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
| `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
| `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | 26 | 0 | security | confinement | route/api, security, filesystem, async/threading |
| `tests/test_run_focus.py` | 492 | 55 | 52 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
| `tests/test_agent_loop.py` | 500 | 54 | 54 | 5 | uncategorized | agent_loop | db/session, import-state |
| `tests/test_manage_mcp_command_allowlist.py` | 168 | 45 | 10 | 0 | services | mcp | db/session, subprocess/script, async/threading, ui/static |
| `tests/test_provider_classification.py` | 115 | 42 | 9 | 2 | services | provider | - |
| `tests/test_review_regressions.py` | 1429 | 41 | 41 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_chat_helpers.py` | 582 | 37 | 24 | 0 | uncategorized | chat_helpers | route/api, filesystem, async/threading |
| `tests/test_model_context.py` | 314 | 36 | 34 | 4 | uncategorized | model_context | db/session, import-state |
| `tests/test_email_oauth.py` | 798 | 34 | 29 | 0 | services | email | route/api, db/session, security, async/threading |
| `tests/test_provider_endpoints.py` | 241 | 58 | 18 | 1 | services | provider | subprocess/script |
| `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
| `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
| `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
| `tests/test_llm_core_temperature.py` | 196 | 41 | 17 | 0 | services | llm | - |
| `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
| `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | 6 | 0 | services | llm | db/session |
| `tests/test_redos_verdict_continuation.py` | 80 | 30 | 5 | 0 | uncategorized | redos_verdict_continuation | route/api |
| `tests/test_chat_helpers.py` | 264 | 31 | 18 | 0 | uncategorized | chat_helpers | route/api |
| `tests/test_provider_detection.py` | 148 | 31 | 31 | 5 | services | provider | - |
| `tests/test_model_context.py` | 251 | 30 | 30 | 4 | uncategorized | model_context | db/session, import-state |
| `tests/test_endpoint_resolver.py` | 148 | 30 | 30 | 6 | uncategorized | endpoint_resolver | - |
| `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
| `tests/test_upload_limits_centralized.py` | 110 | 29 | 5 | 0 | uncategorized | upload_limits_centralized | import-state, filesystem |
| `tests/test_cookbook_docker_access.py` | 339 | 28 | 13 | 0 | services | cookbook | route/api, filesystem, async/threading |
| `tests/test_rename_user_owner_sync.py` | 740 | 27 | 27 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_taxonomy.py` | 151 | 27 | 17 | 0 | uncategorized | taxonomy | security, ui/static |
| `tests/test_endpoint_resolver_urls.py` | 95 | 27 | 23 | 3 | uncategorized | endpoint_resolver_urls | - |
| `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
| `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
| `tests/test_workspace_confine.py` | 527 | 25 | 25 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
| `tests/test_chat_route_tool_policy.py` | 344 | 25 | 23 | 0 | routes | route | filesystem, ui/static |
| `tests/test_tool_path_confinement.py` | 303 | 25 | 25 | 0 | security | confinement | import-state, filesystem, async/threading |
| `tests/test_copilot.py` | 182 | 25 | 18 | 0 | uncategorized | copilot | - |
| `tests/test_llm_core_temperature_reasoning.py` | 133 | 25 | 9 | 0 | services | llm | - |
| `tests/test_provider_endpoints_url_building.py` | 86 | 25 | 3 | 0 | services | provider | - |
| `tests/test_bg_job_tools.py` | 174 | 23 | 16 | 0 | uncategorized | bg_job_tools | filesystem, async/threading |
| `tests/test_taxonomy.py` | 145 | 26 | 16 | 0 | uncategorized | taxonomy | security, ui/static |
| `tests/test_tool_path_confinement.py` | 282 | 24 | 24 | 0 | security | confinement | import-state, filesystem, async/threading |
| `tests/test_copilot.py` | 170 | 23 | 16 | 0 | uncategorized | copilot | - |
| `tests/test_research_utils.py` | 97 | 23 | 23 | 2 | services | research | - |
| `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_tool_support_heuristic.py` | 166 | 22 | 22 | 3 | uncategorized | tool_support_heuristic | - |
| `tests/test_platform_compat.py` | 318 | 21 | 21 | 0 | uncategorized | platform_compat | import-state, filesystem, subprocess/script |
## Top files by physical line count
| File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
|---|---:|---:|---:|---:|---|---|---|
| `tests/test_model_routes.py` | 2099 | 163 | 134 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
| `tests/test_security_regressions.py` | 1515 | 98 | 74 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
| `tests/test_review_regressions.py` | 1429 | 41 | 41 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
| `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
| `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
| `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
| `tests/test_cookbook_helpers.py` | 958 | 69 | 69 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
| `tests/test_upload_handler_cleanup.py` | 831 | 13 | 13 | 0 | uncategorized | upload_handler_cleanup | route/api, db/session, filesystem, async/threading |
| `tests/test_email_oauth.py` | 798 | 34 | 29 | 0 | services | email | route/api, db/session, security, async/threading |
| `tests/test_rename_user_owner_sync.py` | 740 | 27 | 27 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_email_owner_scope.py` | 656 | 13 | 13 | 0 | security | owner_scope | route/api, db/session, filesystem, async/threading |
| `tests/test_model_capability_readers.py` | 646 | 20 | 20 | 0 | uncategorized | model_capability_readers | - |
| `tests/test_chat_helpers.py` | 582 | 37 | 24 | 0 | uncategorized | chat_helpers | route/api, filesystem, async/threading |
| `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
| `tests/test_api_token_routes.py` | 578 | 17 | 17 | 0 | routes | api_routes | route/api, db/session, import-state, async/threading |
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | 26 | 0 | security | confinement | route/api, security, filesystem, async/threading |
| `tests/test_shell_routes.py` | 538 | 69 | 53 | 9 | routes | routes | route/api, import-state, filesystem |
| `tests/test_workspace_confine.py` | 527 | 25 | 25 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
| `tests/test_agent_loop.py` | 500 | 54 | 54 | 5 | uncategorized | agent_loop | db/session, import-state |
| `tests/test_endpoint_probing.py` | 497 | 61 | 36 | 7 | uncategorized | endpoint_probing | route/api, db/session, import-state |
| `tests/test_run_focus.py` | 492 | 55 | 52 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
| `tests/test_tool_policy.py` | 475 | 18 | 18 | 0 | uncategorized | tool_policy | import-state, async/threading |
| `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
| `tests/test_email_owner_scope.py` | 474 | 9 | 9 | 0 | security | owner_scope | route/api, db/session, filesystem, async/threading |
| `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
| `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
| `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | 8 | 0 | uncategorized | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading |
| `tests/test_email_test_connection_oauth.py` | 436 | 9 | 7 | 0 | services | email | route/api, db/session, async/threading |
| `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
| `tests/test_endpoint_owner_scope_followup.py` | 414 | 11 | 11 | 0 | security | owner_scope | route/api, db/session, filesystem |
| `tests/test_api_chat_security.py` | 404 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
| `tests/test_imap_leak_fixes.py` | 404 | 15 | 15 | 0 | uncategorized | imap_leak_fixes | route/api, db/session, security, filesystem |
| `tests/test_companion_readonly.py` | 402 | 17 | 17 | 0 | uncategorized | companion_readonly | db/session, import-state |
| `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_upload_handler_atomicity.py` | 401 | 9 | 9 | 0 | uncategorized | upload_handler_atomicity | filesystem, async/threading |
| `tests/test_auth_regressions.py` | 370 | 15 | 15 | 0 | security | auth | route/api, db/session, import-state, async/threading |
| `tests/test_calendar_recurrence.py` | 356 | 20 | 20 | 0 | services | calendar | - |
| `tests/test_task_cookbook_admin_gate.py` | 350 | 10 | 10 | 0 | services | cookbook | route/api, db/session, import-state, filesystem, async/threading |
| `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
| `tests/test_auth_regressions.py` | 375 | 15 | 15 | 0 | security | auth | route/api, db/session, import-state, async/threading |
| `tests/test_calendar_owner_scope.py` | 345 | 7 | 7 | 0 | security | owner_scope | route/api, db/session, import-state, filesystem, async/threading, ui/static |
| `tests/test_null_owner_gates.py` | 342 | 20 | 20 | 0 | security | owner | route/api, db/session, import-state |
| `tests/test_agent_migration_manifest.py` | 340 | 15 | 15 | 0 | uncategorized | agent_migration_manifest | import-state, filesystem |
| `tests/test_calendar_recurrence.py` | 338 | 19 | 19 | 0 | services | calendar | - |
| `tests/test_tool_policy.py` | 330 | 13 | 13 | 0 | uncategorized | tool_policy | import-state, async/threading |
| `tests/test_workspace_confine.py` | 328 | 18 | 18 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
| `tests/test_diffusion_server_security.py` | 325 | 14 | 14 | 0 | security | security | route/api, import-state, security, filesystem, async/threading, ui/static |
## Split planning candidates
@@ -173,69 +165,53 @@ These are planning candidates only. A later split PR still needs a focused manua
| File | Why included | Setup/risk signals | Suggested handling |
|---|---|---|---|
| `tests/test_model_routes.py` | 2099 lines, 163 collected tests | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_security_regressions.py` | 1515 lines, 98 collected tests | route/api, db/session, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_cookbook_helpers.py` | 958 lines, 69 collected tests | route/api, filesystem, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_shell_routes.py` | 538 lines, 69 collected tests | route/api, import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_endpoint_probing.py` | 497 lines, 61 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_model_routes.py` | 1778 lines, 139 collected tests | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_security_regressions.py` | 1224 lines, 92 collected tests | route/api, db/session, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_provider_classification.py` | 67 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_cookbook_helpers.py` | 912 lines, 65 collected tests | route/api, filesystem, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_shell_routes.py` | 481 lines, 63 collected tests | route/api, import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_pr_blocker_audit.py` | 964 lines, 58 collected tests | import-state, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_research_routes_path_confinement.py` | 565 lines, 58 collected tests | route/api, security, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_run_focus.py` | 492 lines, 55 collected tests | security, filesystem, subprocess/script, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_agent_loop.py` | 500 lines, 54 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_manage_mcp_command_allowlist.py` | 45 collected tests | db/session, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_provider_classification.py` | 42 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_review_regressions.py` | 1429 lines, 41 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_chat_helpers.py` | 582 lines, 37 collected tests | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_model_context.py` | 314 lines, 36 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_email_oauth.py` | 798 lines, 34 collected tests | route/api, db/session, security, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_provider_endpoints.py` | 58 collected tests | subprocess/script | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_agent_loop.py` | 469 lines, 52 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_service_health.py` | 472 lines, 47 collected tests | async/threading | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_run_focus.py` | 399 lines, 47 collected tests | security, filesystem, subprocess/script, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_llm_core_temperature.py` | 41 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_endpoint_probing.py` | 411 lines, 34 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_llm_core_anthropic_temp_omit.py` | 32 collected tests | db/session | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_redos_verdict_continuation.py` | 30 collected tests | route/api | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_chat_helpers.py` | 31 collected tests | route/api | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_provider_detection.py` | 31 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_model_context.py` | 30 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_endpoint_resolver.py` | 30 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_embedding_lanes.py` | 1104 lines, 29 collected tests | filesystem | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_upload_limits_centralized.py` | 29 collected tests | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_cookbook_docker_access.py` | 339 lines, 28 collected tests | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_rename_user_owner_sync.py` | 740 lines, 27 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_taxonomy.py` | 27 collected tests | security, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_endpoint_resolver_urls.py` | 27 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_email_oauth.py` | 580 lines, 28 collected tests | route/api, db/session, security, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_review_regressions.py` | 930 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_rename_user_owner_sync.py` | 686 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_helpers_import_state.py` | 426 lines, 26 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_workspace_confine.py` | 527 lines, 25 collected tests | route/api, filesystem, subprocess/script, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_chat_route_tool_policy.py` | 344 lines, 25 collected tests | filesystem, ui/static | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_tool_path_confinement.py` | 303 lines, 25 collected tests | import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_copilot.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_llm_core_temperature_reasoning.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_provider_endpoints_url_building.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_bg_job_tools.py` | 23 collected tests | filesystem, async/threading | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_taxonomy.py` | 26 collected tests | security, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_tool_path_confinement.py` | 24 collected tests | import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_copilot.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_research_utils.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_api_chat_security.py` | 404 lines, 22 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_api_chat_security.py` | 401 lines, 22 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_tool_support_heuristic.py` | 22 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_platform_compat.py` | 322 lines, 21 collected tests | import-state, filesystem, subprocess/script | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_platform_compat.py` | 318 lines, 21 collected tests | import-state, filesystem, subprocess/script | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_context_compactor.py` | 21 collected tests | db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_prompt_security.py` | 21 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_fenced_inline_args.py` | 21 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_cache_affinity_local_only.py` | 21 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_model_capability_readers.py` | 646 lines, 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_calendar_recurrence.py` | 356 lines, 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_null_owner_gates.py` | 332 lines, 20 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_gallery_endpoint_hardening.py` | 326 lines, 20 collected tests | route/api, security, filesystem, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_redos_llm_parsers.py` | 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
| `tests/test_code_nav_tools.py` | 20 collected tests | db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_null_owner_gates.py` | 342 lines, 20 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_youtube_handler_consolidation.py` | 20 collected tests | route/api, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_tool_policy.py` | 475 lines | import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_fenced_example_not_executed_for_native_models.py` | 345 lines | async/threading | Plan split boundaries before editing. |
| `tests/test_calendar_recurrence.py` | 338 lines | No obvious setup signals from static scan. | Plan split boundaries before editing. |
| `tests/test_workspace_confine.py` | 328 lines | route/api, filesystem, subprocess/script, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_api_token_routes.py` | 578 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_companion_readonly.py` | 402 lines | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_set_admin.py` | 317 lines | route/api, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_llm_core_ollama.py` | 324 lines | async/threading | Plan split boundaries before editing. |
| `tests/test_imap_leak_fixes.py` | 404 lines | route/api, db/session, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_auth_regressions.py` | 370 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_auth_regressions.py` | 375 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_agent_migration_manifest.py` | 340 lines | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_mcp_email_decode_header_spaces.py` | 327 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_diffusion_server_security.py` | 325 lines | route/api, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_upload_handler_cleanup.py` | 831 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_email_owner_scope.py` | 656 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_upload_routes_owner_scope.py` | 344 lines | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_tool_policy.py` | 330 lines | import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_endpoint_owner_scope_followup.py` | 414 lines | route/api, db/session, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_composer_arrow_up_recall_js.py` | 308 lines | filesystem, subprocess/script, ui/static | Plan split boundaries before editing. |
| `tests/test_task_cookbook_admin_gate.py` | 350 lines | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_email_test_connection_oauth.py` | 436 lines | route/api, db/session, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_upload_routes_owner_scope.py` | 315 lines | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_email_owner_scope.py` | 474 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_upload_handler_atomicity.py` | 401 lines | filesystem, async/threading | Plan split boundaries before editing. |
| `tests/test_kv_cache_invalidation_2927.py` | 463 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
| `tests/test_calendar_owner_scope.py` | 345 lines | route/api, db/session, import-state, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
@@ -250,35 +226,28 @@ Before using any of these files as a split target, first decide whether the taxo
| File | Lines | Collected tests | Sub-area | Signals | Suggested follow-up |
|---|---:|---:|---|---|---|
| `tests/test_endpoint_probing.py` | 497 | 61 | endpoint_probing | route/api, db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_pr_blocker_audit.py` | 964 | 58 | pr_blocker_audit | import-state, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_run_focus.py` | 492 | 55 | run_focus | security, filesystem, subprocess/script, ui/static | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_agent_loop.py` | 500 | 54 | agent_loop | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_review_regressions.py` | 1429 | 41 | review_regressions | route/api, db/session, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_chat_helpers.py` | 582 | 37 | chat_helpers | route/api, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_model_context.py` | 314 | 36 | model_context | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_redos_verdict_continuation.py` | 80 | 30 | redos_verdict_continuation | route/api | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_agent_loop.py` | 469 | 52 | agent_loop | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_service_health.py` | 472 | 47 | service_health | async/threading | Review taxonomy mapping before using as a split target. |
| `tests/test_run_focus.py` | 399 | 47 | run_focus | security, filesystem, subprocess/script, ui/static | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_endpoint_probing.py` | 411 | 34 | endpoint_probing | route/api, db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_chat_helpers.py` | 264 | 31 | chat_helpers | route/api | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_model_context.py` | 251 | 30 | model_context | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_endpoint_resolver.py` | 148 | 30 | endpoint_resolver | - | Review taxonomy mapping before using as a split target. |
| `tests/test_upload_limits_centralized.py` | 110 | 29 | upload_limits_centralized | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_taxonomy.py` | 151 | 27 | taxonomy | security, ui/static | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_endpoint_resolver_urls.py` | 95 | 27 | endpoint_resolver_urls | - | Review taxonomy mapping before using as a split target. |
| `tests/test_workspace_confine.py` | 527 | 25 | workspace_confine | route/api, filesystem, subprocess/script, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_copilot.py` | 182 | 25 | copilot | - | Review taxonomy mapping before using as a split target. |
| `tests/test_bg_job_tools.py` | 174 | 23 | bg_job_tools | filesystem, async/threading | Review taxonomy mapping before using as a split target. |
| `tests/test_review_regressions.py` | 930 | 26 | review_regressions | route/api, db/session, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_taxonomy.py` | 145 | 26 | taxonomy | security, ui/static | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_copilot.py` | 170 | 23 | copilot | - | Review taxonomy mapping before using as a split target. |
| `tests/test_tool_support_heuristic.py` | 166 | 22 | tool_support_heuristic | - | Review taxonomy mapping before using as a split target. |
| `tests/test_platform_compat.py` | 322 | 21 | platform_compat | import-state, filesystem, subprocess/script | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_platform_compat.py` | 318 | 21 | platform_compat | import-state, filesystem, subprocess/script | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_context_compactor.py` | 233 | 21 | context_compactor | db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_fenced_inline_args.py` | 186 | 21 | fenced_inline_args | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_cache_affinity_local_only.py` | 104 | 21 | cache_affinity_local_only | - | Review taxonomy mapping before using as a split target. |
| `tests/test_model_capability_readers.py` | 646 | 20 | model_capability_readers | - | Review taxonomy mapping before using as a split target. |
| `tests/test_code_nav_tools.py` | 198 | 20 | code_nav_tools | db/session, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_youtube_handler_consolidation.py` | 104 | 20 | youtube_handler_consolidation | route/api, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_tool_policy.py` | 475 | 18 | tool_policy | import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_fenced_example_not_executed_for_native_models.py` | 345 | 18 | fenced_example_not_executed_for_native_models | async/threading | Review taxonomy mapping before using as a split target. |
| `tests/test_workspace_confine.py` | 328 | 18 | workspace_confine | route/api, filesystem, subprocess/script, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_companion_readonly.py` | 402 | 17 | companion_readonly | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_set_admin.py` | 317 | 17 | set_admin | route/api, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_imap_leak_fixes.py` | 404 | 15 | imap_leak_fixes | route/api, db/session, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_agent_migration_manifest.py` | 340 | 15 | agent_migration_manifest | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_upload_handler_cleanup.py` | 831 | 13 | upload_handler_cleanup | route/api, db/session, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_tool_policy.py` | 330 | 13 | tool_policy | import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
| `tests/test_upload_handler_atomicity.py` | 401 | 9 | upload_handler_atomicity | filesystem, async/threading | Review taxonomy mapping before using as a split target. |
| `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
@@ -290,16 +259,14 @@ Files still in the `uncategorized` taxonomy area are listed separately below so
| File | Lines | Collected tests | Area | Sub-area | Signals | Why this is a candidate |
|---|---:|---:|---|---|---|---|
| `tests/test_provider_classification.py` | 115 | 42 | services | provider | - | 42 collected tests |
| `tests/test_chat_route_tool_policy.py` | 344 | 25 | routes | route | filesystem, ui/static | 344 lines, 25 collected tests |
| `tests/test_llm_core_temperature_reasoning.py` | 133 | 25 | services | llm | - | 25 collected tests |
| `tests/test_provider_endpoints_url_building.py` | 86 | 25 | services | provider | - | 25 collected tests |
| `tests/test_provider_classification.py` | 188 | 67 | services | provider | - | 67 collected tests |
| `tests/test_provider_endpoints.py` | 241 | 58 | services | provider | subprocess/script | 58 collected tests |
| `tests/test_llm_core_temperature.py` | 196 | 41 | services | llm | - | 41 collected tests |
| `tests/test_provider_detection.py` | 148 | 31 | services | provider | - | 31 collected tests |
| `tests/test_embedding_lanes.py` | 1104 | 29 | services | embedding | filesystem | 1104 lines, 29 collected tests |
| `tests/test_research_utils.py` | 97 | 23 | services | research | - | 23 collected tests |
| `tests/test_prompt_security.py` | 203 | 21 | security | security | - | 21 collected tests |
| `tests/test_calendar_recurrence.py` | 356 | 20 | services | calendar | - | 356 lines, 20 collected tests |
| `tests/test_redos_llm_parsers.py` | 201 | 20 | services | llm | - | 20 collected tests |
| `tests/test_llm_core_ollama.py` | 324 | 16 | services | llm | async/threading | 324 lines |
| `tests/test_composer_arrow_up_recall_js.py` | 308 | 11 | js | js | filesystem, subprocess/script, ui/static | 308 lines |
| `tests/test_calendar_recurrence.py` | 338 | 19 | services | calendar | - | 338 lines |
## High-risk candidates to defer first
@@ -307,21 +274,21 @@ These files may still be split later, but not as the first implementation slice
| File | Lines | Collected tests | High-risk signals |
|---|---:|---:|---|
| `tests/test_model_routes.py` | 2099 | 163 | db/session, import-state, route/api |
| `tests/test_security_regressions.py` | 1515 | 98 | db/session, import-state, route/api, security |
| `tests/test_cookbook_helpers.py` | 958 | 69 | route/api |
| `tests/test_shell_routes.py` | 538 | 69 | import-state, route/api |
| `tests/test_endpoint_probing.py` | 497 | 61 | db/session, import-state, route/api |
| `tests/test_model_routes.py` | 1778 | 139 | db/session, import-state, route/api |
| `tests/test_security_regressions.py` | 1224 | 92 | db/session, import-state, route/api, security |
| `tests/test_cookbook_helpers.py` | 912 | 65 | route/api |
| `tests/test_shell_routes.py` | 481 | 63 | import-state, route/api |
| `tests/test_pr_blocker_audit.py` | 964 | 58 | import-state, security |
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | route/api, security |
| `tests/test_run_focus.py` | 492 | 55 | security |
| `tests/test_agent_loop.py` | 500 | 54 | db/session, import-state |
| `tests/test_manage_mcp_command_allowlist.py` | 168 | 45 | db/session |
| `tests/test_review_regressions.py` | 1429 | 41 | db/session, import-state, route/api |
| `tests/test_chat_helpers.py` | 582 | 37 | route/api |
| `tests/test_model_context.py` | 314 | 36 | db/session, import-state |
| `tests/test_email_oauth.py` | 798 | 34 | db/session, route/api, security |
| `tests/test_agent_loop.py` | 469 | 52 | db/session, import-state |
| `tests/test_run_focus.py` | 399 | 47 | security |
| `tests/test_endpoint_probing.py` | 411 | 34 | db/session, import-state, route/api |
| `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | db/session |
| `tests/test_chat_helpers.py` | 264 | 31 | route/api |
| `tests/test_model_context.py` | 251 | 30 | db/session, import-state |
| `tests/test_upload_limits_centralized.py` | 110 | 29 | import-state |
| `tests/test_email_oauth.py` | 580 | 28 | db/session, route/api, security |
| `tests/test_review_regressions.py` | 930 | 26 | db/session, import-state, route/api |
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | db/session, import-state, route/api |
## Rules for future split PRs
@@ -346,7 +313,7 @@ Do not start a split PR from this planning issue alone if the file's boundaries
This document was generated with:
```bash
venv/bin/python tests/tools/build_oversized_test_split_plan.py
.venv/bin/python tests/tools/build_oversized_test_split_plan.py
```
## Freshness check
@@ -354,6 +321,6 @@ venv/bin/python tests/tools/build_oversized_test_split_plan.py
After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:
```bash
venv/bin/python tests/tools/build_oversized_test_split_plan.py
.venv/bin/python tests/tools/build_oversized_test_split_plan.py
git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md
```
@@ -0,0 +1,15 @@
"""Regression coverage for SMTP security saved before Google OAuth."""
from pathlib import Path
_REPO = Path(__file__).resolve().parents[1]
def test_email_tab_oauth_connect_persists_selected_smtp_security():
source = (_REPO / "static" / "js" / "settings.js").read_text(encoding="utf-8")
start = source.index("el('eaf-oauth-btn').addEventListener")
handler_body = source[start:source.index("if (!body.name)", start)]
assert "smtp_security: el('eaf-smtp-security').value" in handler_body
assert "display_name: el('eaf-display-name').value.trim()" in handler_body
@@ -0,0 +1,19 @@
"""Regression coverage for the settings UI after Google OAuth redirects."""
from pathlib import Path
_REPO = Path(__file__).resolve().parents[1]
def test_oauth_redirect_uses_the_module_local_settings_api():
source = (_REPO / "static" / "js" / "settings.js").read_text(encoding="utf-8")
handler = source[
source.index("(function _handleOauthRedirect"):
source.index("const settingsModule =")
]
assert "open('integrations');" in handler
assert "window.settingsModule" not in handler
assert "window.__odysseusAppStarted" not in handler
assert "document.addEventListener('DOMContentLoaded', _showResult, { once: true })" in handler
+86
View File
@@ -0,0 +1,86 @@
"""Regression coverage for issue-description label lifecycle events."""
import json
import shutil
import subprocess
from pathlib import Path
import pytest
_REPO = Path(__file__).resolve().parent.parent
_CHECKER = _REPO / ".github" / "scripts" / "check-issue-description.js"
_WORKFLOW = _REPO / ".github" / "workflows" / "issue-description-check.yml"
pytestmark = pytest.mark.skipif(not shutil.which("node"), reason="node not on PATH")
def _run_closed_issue(action):
harness = r"""
const checkIssueDescription = require(process.argv[1]);
const action = process.argv[2];
const calls = [];
const unexpected = (name) => async () => {
throw new Error(`${name} should not be called for a closed issue`);
};
const github = {
rest: {
issues: {
removeLabel: async (params) => calls.push({ method: 'removeLabel', params }),
getLabel: unexpected('getLabel'),
addLabels: unexpected('addLabels'),
listComments: unexpected('listComments'),
createComment: unexpected('createComment'),
updateComment: unexpected('updateComment'),
deleteComment: unexpected('deleteComment'),
},
},
};
const context = {
payload: {
action,
issue: { number: 42, state: 'closed', body: '', labels: [] },
},
repo: { owner: 'odysseus-dev', repo: 'odysseus' },
};
const core = {
warning: unexpected('core.warning'),
setFailed: unexpected('core.setFailed'),
};
checkIssueDescription({ github, context, core })
.then(() => process.stdout.write(JSON.stringify(calls)))
.catch((error) => {
console.error(error);
process.exitCode = 1;
});
"""
proc = subprocess.run(
["node", "-e", harness, str(_CHECKER), action],
capture_output=True,
text=True,
cwd=str(_REPO),
timeout=30,
)
assert proc.returncode == 0, proc.stderr
return json.loads(proc.stdout)
def test_workflow_handles_issue_closures():
workflow = _WORKFLOW.read_text()
assert "types: [opened, edited, reopened, closed]" in workflow
@pytest.mark.parametrize("action", ["closed", "edited"])
def test_closed_issue_only_drops_ready_for_review(action):
assert _run_closed_issue(action) == [
{
"method": "removeLabel",
"params": {
"owner": "odysseus-dev",
"repo": "odysseus",
"issue_number": 42,
"name": "ready for review",
},
}
]
+25 -1
View File
@@ -29,6 +29,13 @@ from src.llm_core import _anthropic_rejects_temperature, _build_anthropic_payloa
"anthropic/claude-opus-4-7", # tolerate a provider-prefixed id
"claude-opus-4-10", # future minor still >= 4.7
"claude-opus-5-0", # future major
# Major-only ids: a missing minor reads as `.0`, so these are >= 4.7 too
# (issue #5753). Before the fix the version pattern required a minor, so
# these fell through to "accepts temperature" and every call 400'd.
"claude-opus-5",
"claude-opus-5-20260101", # major-only + dated snapshot
"anthropic/claude-opus-5", # major-only behind a provider prefix
"claude-opus-6", # future major-only
],
)
def test_opus_47_plus_rejects_temperature(model):
@@ -48,7 +55,10 @@ def test_opus_47_plus_rejects_temperature(model):
"claude-opus-4-6-20251201", # dated 4.6 snapshot — older, still keeps temperature
"claude-sonnet-4-6",
"claude-3-5-sonnet",
"claude-3-opus-20240229", # legacy Claude 3 Opus — no opus-N-M pattern, kept
"claude-3-opus-20240229", # legacy Claude 3 Opus — date directly after
# "opus-", so the major must not swallow it as version 20240229 (that is
# what makes capping the major at 1-2 digits necessary once the minor
# became optional in #5753).
"claude-haiku-4-5",
"claude-x",
"octopus-4-8", # "opus" only as a substring of another word — must not match
@@ -87,6 +97,20 @@ def test_payload_keeps_temperature_for_older_models():
assert _payload("claude-3-5-sonnet", 1.2)["temperature"] == 1.0
def test_payload_omits_temperature_for_major_only_opus_5():
# Issue #5753: the scheduled-task path calls stream_agent_loop() without a
# temperature and inherits its 0.3 default, so `claude-opus-5` 400'd on every
# run and surfaced as "the model returned an empty response". Interactive chat
# leaves temperature None and never hit it.
assert "temperature" not in _payload("claude-opus-5", 0.3)
def test_payload_keeps_temperature_for_legacy_claude_3_opus():
# Guards the major-digit cap: `opus-20240229` must not parse as version
# 20240229, or Claude 3 Opus would silently lose the caller's temperature.
assert _payload("claude-3-opus-20240229", 0.5)["temperature"] == 0.5
def test_payload_keeps_temperature_for_dated_opus_4_0():
# Anthropic's dated id for Opus 4.0 (claude-opus-4-20250514) is in this repo's
# ANTHROPIC_MODELS list. The date must not be misread as a >= 4.7 minor, or the
+44
View File
@@ -214,6 +214,50 @@ def test_inline_code_content_is_html_escaped(node_available):
assert "<b>" not in html
def test_fenced_code_keeps_dollar_ampersand(node_available):
# Issue #5663: the block-restore pass used a string replacement, so `$&` in a
# restored block was read as "the matched text" and re-inserted the
# placeholder. `perl -pe 's/world/$& again/'` rendered as
# "s/world/___CODE_BLOCK_0___amp; again/" — the trailing "amp;" is the orphan
# left behind after `$&` consumed the `$&` of the escaped `$&amp;`.
html = _run_markdown_case(
"```sh\necho \"hello world\" | perl -pe 's/world/$& again/'\n```"
)
assert "___CODE_BLOCK_" not in html
assert "s/world/$&amp; again/" in html
assert "amp; again" not in html.replace("$&amp; again", "")
def test_fenced_code_keeps_dollar_backtick_and_quote(node_available):
# `` $` `` and `$'` splice the text before/after the placeholder into the
# block. Unlike `$&` these leave no placeholder behind — the characters just
# vanish — so assert the content survives verbatim.
html = _run_markdown_case("```sh\nsed \"s/$`/x/\" && sed \"s/$'/y/\"\n```")
assert "___CODE_BLOCK_" not in html
assert "s/$`/x/" in html
assert "s/$&#39;/y/" in html
def test_fenced_code_keeps_double_dollar(node_available):
# `$$` collapsed to a single `$` in the restored block.
html = _run_markdown_case('```sh\necho "$$USD and $$"\n```')
assert "$$USD and $$" in html
def test_mermaid_block_keeps_dollar_ampersand(node_available):
# The mermaid restore site had the same hazard: a node label containing `$&`
# re-inserted the ___MERMAID_BLOCK_n___ placeholder into the diagram source,
# which then fails to parse. The math and allowed-HTML sites are fixed the
# same way; they need KaTeX/sanitizer conditions this harness doesn't set up.
html = _run_markdown_case('```mermaid\ngraph TD; A["$&"] --> B;\n```')
assert "___MERMAID_BLOCK_" not in html
assert "$&amp;" in html
def test_currency_dollar_amounts_are_not_rendered_as_math(node_available):
# "$5 to $10" used to pair the two dollar signs as inline-math delimiters
# and render "5 to" through KaTeX. Pandoc-style rules now reject it: the
@@ -0,0 +1,15 @@
"""Regression coverage for the built-in MCP servers' SDK compatibility line."""
from pathlib import Path
REQUIREMENTS = Path(__file__).resolve().parents[1] / "requirements.txt"
def test_mcp_requirement_excludes_breaking_v2_sdk():
requirements = [
line.split("#", 1)[0].strip().replace(" ", "")
for line in REQUIREMENTS.read_text(encoding="utf-8").splitlines()
]
assert "mcp<2" in requirements
@@ -1,8 +1,8 @@
"""Provider endpoint URL-building tests.
Covers ``build_chat_url`` and ``build_models_url`` for the provider families
exercised by this module: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI,
DeepSeek, and Ollama (local and cloud).
Covers ``build_chat_url`` and ``build_models_url`` for every provider named in
ROADMAP.md: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, DeepSeek, Ollama
(local + cloud).
"""
import pytest
+11
View File
@@ -0,0 +1,11 @@
"""Regression test for the search route shim (slice 2j, #4082/#4071)."""
import importlib
import routes.search_routes as _shim_search # noqa: F401
def test_legacy_and_canonical_search_module_are_same_object():
legacy = importlib.import_module("routes.search_routes")
canonical = importlib.import_module("routes.search.search_routes")
assert legacy is canonical
+58
View File
@@ -0,0 +1,58 @@
"""Regression for issue #5697 — skill timestamps must not use ``datetime.utcnow()``.
``_now_iso()`` builds the ``created`` value in skill frontmatter. ``utcnow()``
returns a *naive* datetime and has been deprecated since Python 3.12, scheduled
for removal. The replacement must stay timezone-aware while keeping the
serialized ``YYYY-MM-DDTHH:MM:SSZ`` shape, so skill files written by older
versions keep parsing.
The UTC check matters on its own: a bare ``datetime.now()`` also produces the
right shape, but emits local wall time, which would silently backdate or
postdate skills for every user outside UTC.
"""
import os
import re
import time
import warnings
from datetime import datetime, timezone
import pytest
from services.memory.skill_format import _now_iso
_ISO_Z = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$")
def test_now_iso_keeps_serialized_shape():
assert _ISO_Z.match(_now_iso())
def test_now_iso_emits_no_deprecation_warning():
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
_now_iso()
assert not [w for w in caught if issubclass(w.category, DeprecationWarning)]
@pytest.mark.skipif(
not hasattr(time, "tzset"),
reason="time.tzset is unavailable on this platform",
)
def test_now_iso_is_utc_not_local_time():
"""Pin UTC under a non-UTC local timezone, where the two visibly diverge."""
original_tz = os.environ.get("TZ")
os.environ["TZ"] = "Asia/Amman" # UTC+3, never UTC
time.tzset()
try:
emitted = datetime.strptime(_now_iso(), "%Y-%m-%dT%H:%M:%SZ").replace(
tzinfo=timezone.utc
)
drift = abs((emitted - datetime.now(timezone.utc)).total_seconds())
assert drift < 60, f"timestamp is {drift}s off UTC — local time leaked in"
finally:
if original_tz is None:
os.environ.pop("TZ", None)
else:
os.environ["TZ"] = original_tz
time.tzset()
@@ -0,0 +1,97 @@
import os
import time
from pathlib import Path
import pytest
# Adjust the import path if your file is directly in ./services instead of ./services/tts
from services.tts.tts_service import TTSService
def test_cache_under_limit(tmp_path, monkeypatch):
"""Test that writing a file under the size limit does not trigger eviction."""
# Set a tiny limit: 100 bytes
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "100")
# Initialize service with pytest's temporary directory
service = TTSService(cache_dir=str(tmp_path))
# Write a 40-byte file (under the 100-byte limit)
service._put_cache("test_key", b"x" * 40)
# Verify the file was written and nothing was deleted
files = list(tmp_path.glob("*.*"))
assert len(files) == 1
assert sum(f.stat().st_size for f in files) == 40
def test_cache_exceeds_limit_triggers_eviction(tmp_path, monkeypatch):
"""Test that exceeding the limit evicts the oldest files down to 80% capacity."""
# Set limit to 100 bytes. 80% target capacity will be 80 bytes.
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "100")
service = TTSService(cache_dir=str(tmp_path))
# 1. Setup: Manually create two older files (40 bytes each)
file1 = tmp_path / "oldest.wav"
file2 = tmp_path / "middle.wav"
file1.write_bytes(b"a" * 40)
file2.write_bytes(b"b" * 40)
# Spoof timestamps so file1 is explicitly older than file2
now = time.time()
os.utime(file1, (now - 100, now - 100)) # 100 seconds ago
os.utime(file2, (now - 50, now - 50)) # 50 seconds ago
# 2. Action: Write a 3rd file using the service method (40 bytes)
# Total cache is now 120 bytes, which exceeds 100.
# It should delete oldest (file1) to drop to 80 bytes (which matches the 80% target).
service._put_cache("newest", b"c" * 40)
# 3. Assertions
# The newest file should exist (saved as .wav because it lacks MP3 magic bytes)
newest_file = tmp_path / "newest.wav"
assert not file1.exists(), "The oldest file should have been evicted."
assert file2.exists(), "The middle file should still exist."
assert newest_file.exists(), "The newest file should have been saved."
# Verify the final directory size is <= 80 bytes
total_size = sum(f.stat().st_size for f in tmp_path.glob("*.*"))
assert total_size <= 80
def test_cache_limit_disabled(tmp_path, monkeypatch):
"""Test that setting max bytes to 0 disables eviction."""
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "0")
service = TTSService(cache_dir=str(tmp_path))
# Write 3 large files that would normally trigger eviction
service._put_cache("file1", b"x" * 1000)
service._put_cache("file2", b"x" * 1000)
service._put_cache("file3", b"x" * 1000)
# Ensure nothing was deleted
files = list(tmp_path.glob("*.*"))
assert len(files) == 3
assert sum(f.stat().st_size for f in files) == 3000
def test_cache_eviction_handles_unlink_error_gracefully(tmp_path, monkeypatch):
"""Test that if unlinking a file fails, _put_cache still succeeds without raising."""
service = TTSService(cache_dir=str(tmp_path))
service.max_cache_bytes = 50
# Create a file to evict
old_file = tmp_path / "old.wav"
old_file.write_bytes(b"x" * 40)
# Monkeypatch unlink on Path objects to simulate a PermissionError / file-lock failure
def mock_unlink(self_path):
raise OSError("Permission denied / file locked")
monkeypatch.setattr(Path, "unlink", mock_unlink)
# Writing a new file triggers eviction which encounters the mocked unlink error
try:
service._put_cache("new_key", b"y" * 40)
except Exception as e:
pytest.fail(f"_put_cache raised an exception during failed eviction: {e}")
# The new file should still be written successfully
assert (tmp_path / "new_key.wav").exists()
+2 -10
View File
@@ -441,14 +441,6 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
lines = [
"# Oversized Test File Split Plan",
"",
"> [!NOTE]",
"> This is generated planning material based on the current test tree.",
">",
"> It is non-canonical and disposable. Durable testing policy belongs in",
"> `tests/TESTING_STANDARD.md`; active split selection and prioritization",
"> belong in approved issues or the project tracker.",
"> It does not define runtime behaviour or approve any test-file split.",
"",
"## Purpose",
"",
"This document plans future oversized test-file splits using current repo data.",
@@ -516,7 +508,7 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
"This document was generated with:",
"",
"```bash",
"venv/bin/python tests/tools/build_oversized_test_split_plan.py",
".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
"```",
"",
"## Freshness check",
@@ -524,7 +516,7 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
"After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:",
"",
"```bash",
"venv/bin/python tests/tools/build_oversized_test_split_plan.py",
".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
"git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md",
"```",
"",