mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-08-02 03:28:36 -04:00
Compare commits
11 Commits
dev
..
7b4c83089b
| Author | SHA1 | Date | |
|---|---|---|---|
| 7b4c83089b | |||
| 6bc6ee9c7e | |||
| d190980f47 | |||
| 8670aed9ad | |||
| c762efe1c9 | |||
| 7fe3bc6482 | |||
| ab6f3fa70b | |||
| 69041b5f50 | |||
| 2ac72d5964 | |||
| 91764ad3e7 | |||
| c7c1b7717e |
@@ -189,7 +189,6 @@ SEARXNG_INSTANCE=http://localhost:8080
|
||||
# ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=26214400 # email compose attachment (25 MB)
|
||||
# ODYSSEUS_STT_MAX_AUDIO_BYTES=26214400 # speech-to-text audio (25 MB)
|
||||
# ODYSSEUS_ICS_MAX_BYTES=10485760 # calendar .ics import (10 MB)
|
||||
# ODYSSEUS_TTS_CACHE_MAX_BYTES=524288000 # TTS cache (500 MB)
|
||||
|
||||
# ============================================================
|
||||
# Host Docker access (explicit opt-in)
|
||||
|
||||
@@ -8,8 +8,8 @@ body:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
and [discussions](https://github.com/odysseus-dev/odysseus/discussions) first.
|
||||
Feature requests that duplicate [ROADMAP.md](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md)
|
||||
or an existing open issue will be closed as duplicates.
|
||||
The [roadmap](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md) is directional rather than a complete backlog.
|
||||
Feature requests that duplicate an existing issue or accepted proposal may be closed as duplicates.
|
||||
|
||||
If your idea needs community input before it becomes a concrete proposal,
|
||||
start a [discussion](https://github.com/odysseus-dev/odysseus/discussions/categories/ideas) instead.
|
||||
|
||||
@@ -153,16 +153,6 @@ module.exports = async ({ github, context, core }) => {
|
||||
}
|
||||
}
|
||||
|
||||
const LABEL_BAD = 'needs more info';
|
||||
const LABEL_GOOD = 'ready for review';
|
||||
|
||||
// Closed issues are no longer awaiting review.
|
||||
// This also prevents later edits to closed issues from restoring the label.
|
||||
if (issue.state === 'closed') {
|
||||
await dropLabel(LABEL_GOOD);
|
||||
return;
|
||||
}
|
||||
|
||||
// ── Find existing bot comment to update in-place ──────────────────────────
|
||||
const MARKER = '<!-- issue-description-check -->';
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
@@ -170,6 +160,9 @@ module.exports = async ({ github, context, core }) => {
|
||||
});
|
||||
const existing = comments.find(c => c.user.type === 'Bot' && c.body.includes(MARKER));
|
||||
|
||||
const LABEL_BAD = 'needs more info';
|
||||
const LABEL_GOOD = 'ready for review';
|
||||
|
||||
if (failures.length === 0) {
|
||||
if (existing) {
|
||||
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
|
||||
|
||||
@@ -2,7 +2,7 @@ name: ci / issue description check
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened, edited, reopened, closed]
|
||||
types: [opened, edited, reopened]
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Odysseus
|
||||
|
||||
<p align="center">
|
||||
<img src="docs/odysseus-wordmark.png" alt="Odysseus" width="238">
|
||||
</p>
|
||||
@@ -9,6 +11,8 @@
|
||||
<p align="center">
|
||||
<a href="#quick-start">Quick Start</a> ·
|
||||
<a href="docs/setup.md">Setup Guide</a> ·
|
||||
<a href="docs/ARCHITECTURE.md">Architecture</a> ·
|
||||
<a href="SECURITY.md">Security</a> ·
|
||||
<a href="CONTRIBUTING.md">Contributing</a> ·
|
||||
<a href="ROADMAP.md">Roadmap</a>
|
||||
</p>
|
||||
@@ -51,15 +55,15 @@ Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration
|
||||
|
||||
## Demo
|
||||
|
||||
A full hover-to-play tour lives on the landing page: [`docs/index.html`](docs/index.html).
|
||||
Explore the interface through the [interactive product tour](docs/index.html).
|
||||
|
||||
## Contributing
|
||||
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, docs, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, documentation, and small focused refactors. Read the [contributing guide](CONTRIBUTING.md), review the [public roadmap](ROADMAP.md), and browse the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues).
|
||||
|
||||
## Security
|
||||
|
||||
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Deployment details are in the [setup guide](docs/setup.md#security-notes).
|
||||
Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model or service ports publicly. Read the [security policy](SECURITY.md) and the [deployment security guidance](docs/setup.md#security-notes).
|
||||
|
||||
## Star History
|
||||
|
||||
@@ -73,4 +77,4 @@ Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled
|
||||
|
||||
## License
|
||||
|
||||
AGPL-3.0-or-later -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
|
||||
Licensed under AGPL-3.0-or-later. See the [license](LICENSE) and [acknowledgments](ACKNOWLEDGMENTS.md).
|
||||
|
||||
+43
-75
@@ -1,87 +1,55 @@
|
||||
# Roadmap / Help Wanted
|
||||
# Roadmap
|
||||
|
||||
Odysseus is on a voyage, but not home yet. It works great for me (lol), but this ship is moving fast and feedback/help would be appreciated! (I don't know what I'm doing, help).
|
||||
This document provides a high-level view of the areas Odysseus is currently improving.
|
||||
|
||||
If you see weird CSS, strange layout behavior, or a suspiciously murky corner of
|
||||
the codebase, you are probably right to stay away.
|
||||
It is directional rather than exhaustive. Priorities may change as the project evolves, defects are discovered, and maintainers learn more from implementation work and user feedback.
|
||||
|
||||
## High Priority
|
||||
For current implementation work, see the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues). Accepted behaviour should be documented in the repository alongside the code.
|
||||
|
||||
- SQUASH BUGS
|
||||
- Fresh install smoke tests on Linux, macOS, and Windows. Docker, native Python,
|
||||
and WSL all need coverage.
|
||||
## Current priorities
|
||||
|
||||
- Integration audit: do integrations even work? Confirm what works, what needs setup docs, and what should be removed or hidden.
|
||||
- Cookbook reliability on other computers. This is probably the area most likely to need work across different machines, GPUs, drivers, shells, and Python environments.
|
||||
- Cookbook SGLang support across platforms. Make sure SGLang setup/serve works
|
||||
predictably on Linux, Windows/WSL, macOS where possible, Docker, and common
|
||||
NVIDIA/AMD hardware paths.
|
||||
- Deep Research model presets by hardware. Recommend approved model/parameter
|
||||
profiles for small, medium, and large local setups so people with different
|
||||
hardware can use Deep Research without guessing. Surface this either in Deep
|
||||
Research settings or as a Cookbook scan/dropdown suggestion.
|
||||
- Cookbook model scan/download ranking. Prioritize newer architectures and
|
||||
better hardware-fit models instead of scoring everything almost the same.
|
||||
Ranking should account for architecture age, quant format, VRAM/RAM fit,
|
||||
backend support, vision/mmproj requirements, and likely serve reliability.
|
||||
- Cookbook error feedback and logging. Failed downloads, dependency installs,
|
||||
preflights, and serve jobs should show the actual command/output/error in the
|
||||
UI, with copyable logs and clear next steps instead of just "crashed".
|
||||
- Agent prompt/context bloat. Agent mode is too heavy for smaller local models:
|
||||
tool schemas, skills, memory, documents, and instructions can eat the context
|
||||
before the user request really starts. We need slimmer prompts, better tool
|
||||
selection, smaller default tool sets, and clearer guidance for models with
|
||||
4k/8k/16k context windows.
|
||||
- Local model speculative decoding support. For Odysseus-tuned local models,
|
||||
plan to ship or recommend a small same-tokenizer draft model when the serving
|
||||
backend supports it. Early vLLM testing showed a generic `Qwen3-0.6B` draft
|
||||
beside `Qwen3-8B` can materially reduce wall time, while an unsupported
|
||||
DSpark conversion performed poorly. Treat this as a supported draft-model lane
|
||||
first; keep MTP-specific packaging as future work only when the architecture
|
||||
and runtime support are real. Judge this by time-to-success, tool correctness,
|
||||
grammar, and unchanged target output, not tokens/sec alone.
|
||||
- Skill/tool prompt-injection audit. User-editable skills, notes, documents,
|
||||
fetched pages, and memories should be treated as untrusted data. Keep testing
|
||||
whether models follow malicious instructions from those surfaces.
|
||||
- Better degraded-state reporting for ChromaDB, SearXNG, email, ntfy, and provider probes.
|
||||
- Email performance audit. Fetching, searching, opening, deleting, and sending
|
||||
email can feel slow, especially over IMAP/SMTP providers with high latency.
|
||||
Need someone who knows mail performance to profile the current flow, identify
|
||||
whether the bottleneck is IMAP folder select/fetch, cache invalidation,
|
||||
attachment/body loading, SMTP handshakes, or frontend refresh behavior, then
|
||||
propose safer caching/prefetch/batching without breaking multi-account state.
|
||||
- Provider setup/probing audit for Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, and DeepSeek.
|
||||
### Reliability and setup
|
||||
|
||||
## Refactor Targets
|
||||
- CSS cleanup. `static/style.css` basically Calypso's island atm.
|
||||
- Tour core helper. The onboarding tours have too much copy-pasted scaffolding; promote a shared `tour-core.js` helper before adding more tours.
|
||||
- Modal/window positioning cleanup. Some window controls have improved, but the
|
||||
underlying popup/dropdown/fixed-position behavior is still too fragile.
|
||||
- Mobile media override discoverability. A lot of "CSS did not move" bugs are mobile `@media` overrides of the same selector; comments or linting around desktop/mobile paired rules would help.
|
||||
- Dead code pass for old routes, stale feature flags, and unused UI states.
|
||||
- Improve fresh-install and smoke-test coverage across supported environments.
|
||||
- Make provider setup, probing, and failure states more predictable.
|
||||
- Improve Cookbook reliability across hardware, operating systems, drivers, shells, and serving backends.
|
||||
- Improve degraded-state reporting and recovery guidance when optional services are unavailable.
|
||||
|
||||
## Frontend
|
||||
### Local model workflows
|
||||
|
||||
- Expand the Editor for quicker, more robust everyday use. Better file/document
|
||||
handling, smoother window behavior, clearer save/export flows, stronger image
|
||||
editing affordances, and fewer brittle edge cases.
|
||||
- Better AI integration for Notes and Todos. Notes should be easier for the
|
||||
agent to read, update, summarize, and turn into actions. Todos should be
|
||||
assignable to an agent from the UI, possibly through a button, task action,
|
||||
or dedicated skill/tool flow.
|
||||
- Mobile gallery/editor polish. Easier to launch/download inpaint model or any missing pieces.
|
||||
- Accessibility pass: keyboard navigation, focus states, contrast, reduced motion.
|
||||
- Improve empty states and error messages on fresh installs.
|
||||
- Tighten first-run setup, hints, and tours so they do not repeat or fight each other.
|
||||
- Vendor CDN assets eventually for a more fully self-hosted/offline mode.
|
||||
- Improve hardware-aware model recommendations and compatibility guidance.
|
||||
- Evaluate serving optimizations, including speculative decoding, through reproducible benchmarks.
|
||||
- Improve installation, preflight checks, logging, and error reporting for local model serving.
|
||||
- Reduce prompt and context overhead for smaller local models.
|
||||
|
||||
## Backend
|
||||
### Safety and resilience
|
||||
|
||||
- More tests around endpoint probing and provider setup.
|
||||
- Better task scheduler defaults and visibility.
|
||||
- Backup/restore guide and helper flow for `data/`.
|
||||
- Security hardening around admin-only tools and clear docs for their risk.
|
||||
- Continue hardening tool execution, filesystem access, credentials, networking, and destructive operations.
|
||||
- Treat content from documents, notes, memories, skills, and fetched pages as potentially untrusted.
|
||||
- Improve security-focused regression coverage and operational guidance.
|
||||
- Review integrations that expand access to sensitive data or privileged operations.
|
||||
|
||||
## Not The Focus Right Now
|
||||
### Product usability
|
||||
|
||||
I prob shouldnt add more themes.
|
||||
- Improve first-run setup, onboarding, hints, and tours.
|
||||
- Improve accessibility, keyboard navigation, focus behaviour, contrast, and reduced-motion support.
|
||||
- Improve empty states, error messages, and recovery paths.
|
||||
- Strengthen Notes, Todos, Editor, mobile, and everyday workspace flows.
|
||||
|
||||
### Architecture and maintainability
|
||||
|
||||
- Reduce duplication and technical debt through focused, reviewable refactors.
|
||||
- Improve subsystem documentation as behaviour and architecture become stable.
|
||||
- Remove stale code, obsolete feature flags, and unsupported integrations.
|
||||
- Keep implementation decisions grounded in current code and verified behaviour.
|
||||
|
||||
## Tracking work
|
||||
|
||||
Concrete implementation tasks, defects, proposals, and technical investigations are tracked in:
|
||||
|
||||
- [GitHub Issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
- [Contributing Guide](CONTRIBUTING.md)
|
||||
|
||||
Maintainers may use additional private coordination tools for ownership, planning, and unresolved decisions.
|
||||
|
||||
This roadmap is not a complete backlog or a guarantee that a particular item will be delivered.
|
||||
|
||||
+3
-1
@@ -37,4 +37,6 @@ Only `.env.example`, docs, source, tests, and static assets should be committed.
|
||||
|
||||
## Reporting
|
||||
|
||||
Please report vulnerabilities privately via GitHub security advisories if available, or by opening a minimal issue that does not disclose exploit details.
|
||||
Report security vulnerabilities privately through [GitHub Security Advisories](https://github.com/odysseus-dev/odysseus/security/advisories/new).
|
||||
|
||||
Do not open a public issue or discussion, and do not disclose exploit details publicly.
|
||||
|
||||
+5
-5
@@ -68,14 +68,14 @@ External content that reaches the LLM is treated as untrusted via `src/prompt_se
|
||||
- `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer` everywhere.
|
||||
- **CSP:** nonce-based `script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net`. `style-src 'unsafe-inline'` is intentionally kept — `static/index.html` ships inline `<style>` blocks and JS modules set `style=""` attributes at runtime. Inline styles do not execute script so the risk is visual-only. Removing this requires templating the HTML files and auditing all JS-set style attributes.
|
||||
|
||||
## Token-Supplied Model Endpoints
|
||||
|
||||
Direct `/api/v1/chat` requests with a token-supplied `base_url` must use a public HTTP(S) endpoint. This restriction applies only to untrusted direct values; administrator-configured endpoints may intentionally use local or LAN URLs for private model providers.
|
||||
|
||||
## Known Gaps
|
||||
|
||||
These are open, acknowledged, and contributor help is welcome:
|
||||
|
||||
1. **No shell/filesystem sandbox.** The agent `bash` and `read_file`/`write_file` tools run as the app process user with no network egress filtering or filesystem confinement. A successful prompt-injection reaching a shell-enabled admin session can make outbound requests to internal services. See #1058 for the sandbox proposal.
|
||||
|
||||
2. **SSRF via `/api/v1/chat` `base_url` parameter.** A chat-scoped API token can supply an arbitrary `base_url`; the server forwards the LLM request to that host without validating the scheme or address. PR #1039 fixes this.
|
||||
|
||||
3. **`src/search/` partial consolidation.** `src.search.core` and `src.search.providers` correctly alias `services.search` via `sys.modules` replacement. `analytics`, `cache`, `content`, `query`, and `ranking` are still independent copies that can drift. The SSRF regression tests in `tests/test_webhook_ssrf_resilience.py` test `src.webhook_manager` directly (separate from search), so the safety net there is intact. See #1058.
|
||||
|
||||
4. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
|
||||
2. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
|
||||
|
||||
@@ -692,7 +692,7 @@ from routes.history.history_routes import setup_history_routes
|
||||
app.include_router(setup_history_routes(session_manager, upload_handler=upload_handler))
|
||||
|
||||
# Search
|
||||
from routes.search.search_routes import setup_search_routes
|
||||
from routes.search_routes import setup_search_routes
|
||||
app.include_router(setup_search_routes(config))
|
||||
|
||||
# Presets
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Odysseus discovery maps
|
||||
|
||||
Compact, code-grounded discovery maps of cross-cutting systems in the checked-in Odysseus codebase. They preserve investigation context and open factual questions; they are not canonical subsystem specifications, a feature certification, or a substitute for normal testing.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Checked-in code is the source of truth for current behaviour. Mature subsystem specifications, where they exist, are the canonical documentation of accepted subsystem behaviour. Check code, tests, and configuration before reconciling a discovery finding. Discovery remains non-canonical.
|
||||
|
||||
## Explore the maps
|
||||
|
||||
| Document | Purpose |
|
||||
|---|---|
|
||||
| [Current system map](system-map.md) | Records evidence locations, confirmed local observations, and factual open questions about subsystem boundaries. |
|
||||
| [Safety boundaries](safety-boundaries.md) | Records evidence about broad authority, safeguards, confirmed risks or gaps, and unverified behaviour. |
|
||||
|
||||
## Working rules
|
||||
|
||||
- **Trace the code first.** Confirm the current path in source before recording a claim.
|
||||
- **Promote selectively.** When an owning mature specification exists, add a fact only when it is verified, useful, and not already represented there.
|
||||
- **Record missing ownership.** When no owning specification exists, retain the verified finding in discovery and record missing documentation ownership as a follow-up.
|
||||
- **Retain uncertainty here.** Keep unresolved questions and useful investigation context in discovery rather than treating them as canonical truth.
|
||||
- **Keep specifications current-state only.** Do not record intentions, design direction, refactor plans, decision history, priority, ownership, or sequencing here.
|
||||
- **Investigate with cause.** Do not exhaustively revalidate existing functionality without a report, visible failure, relevant change, or high-authority review need.
|
||||
- **Review authority carefully.** Give execution, data access, external tools, credentials, destructive operations, and unattended work focused review.
|
||||
- **Use stable locations.** Cite modules, routes, classes, and functions instead of fragile line ranges or generated evidence tables.
|
||||
|
||||
## Reconciliation flow
|
||||
|
||||
1. Start with the relevant map and trace the cited code.
|
||||
2. Classify the finding against current source evidence and an owning mature specification where one exists.
|
||||
3. Promote only verified, useful facts that are missing from an existing owning specification.
|
||||
4. When no owning specification exists, retain the verified finding here and record missing documentation ownership as a follow-up; otherwise retain unresolved context here and correct stale wording.
|
||||
|
||||
> [!NOTE]
|
||||
> This package intentionally contains no generator, validator, maturity scale, feature database, or parallel work tracker. The [architecture runtime inventory](./architecture-runtime-inventory.md) preserves dated structural metrics, investigation context, and historical planning as an explicitly non-canonical snapshot.
|
||||
@@ -0,0 +1,331 @@
|
||||
# Architecture runtime inventory
|
||||
|
||||
> [!WARNING]
|
||||
> This document is a dated structural snapshot, not a canonical runtime specification.
|
||||
> Counts, paths, and implementation details may drift as the repository changes.
|
||||
> Verify implementation-sensitive claims against the current code and tests.
|
||||
|
||||
- **Branch:** `discovery`
|
||||
- **Commit:** `c762efe1c97a`
|
||||
- **Generated:** `2026-07-28T05:38:14+01:00`
|
||||
- **Historical context:** readability and refactor planning in [#4071](https://github.com/odysseus-dev/odysseus/issues/4071) and [#4082](https://github.com/odysseus-dev/odysseus/issues/4082)
|
||||
|
||||
## Disposition
|
||||
|
||||
> [!NOTE]
|
||||
> Reviewed for documentation classification. Stable runtime structure and
|
||||
> subsystem ownership have been transferred to the proposed canonical
|
||||
> destination, [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md), for
|
||||
> maintainer review.
|
||||
>
|
||||
> This document retains dated metrics, rankings, investigation context,
|
||||
> refactor-sensitive observations, and historical planning. Those contents are
|
||||
> non-canonical and belong under `discovery/`.
|
||||
|
||||
| Content | Authority and destination |
|
||||
|---|---|
|
||||
| Stable runtime structure | Proposed canonical destination: `docs/ARCHITECTURE.md` |
|
||||
| Stable subsystem boundaries | Proposed canonical destination: `docs/ARCHITECTURE.md` |
|
||||
| Frontend module organization | `static/js/MODULE_SUMMARY.md` |
|
||||
| Counts, line totals, and rankings | This non-canonical inventory |
|
||||
| Investigation context and open questions | `discovery/` |
|
||||
| Refactor options and prioritization | Issues, Plane, or non-canonical discovery material |
|
||||
|
||||
The transfer preserves the stable facts without promoting generated metrics or
|
||||
historical prioritization into canonical documentation.
|
||||
|
||||
## Purpose
|
||||
|
||||
This inventory provides a reviewable map of the current repository structure,
|
||||
large runtime modules, major subsystem boundaries, and refactor-sensitive areas.
|
||||
|
||||
It does not:
|
||||
|
||||
- define accepted subsystem behaviour;
|
||||
- certify runtime correctness;
|
||||
- prescribe a committed refactor sequence;
|
||||
- replace focused specifications, tests, or source review.
|
||||
|
||||
For cross-cutting implementation evidence, see the [discovery maps](./README.md).
|
||||
|
||||
## Top-level runtime structure
|
||||
|
||||
| Area | Role |
|
||||
|---|---|
|
||||
| `app.py` | FastAPI application composition and entry point |
|
||||
| `launcher.py` | Application launch support |
|
||||
| `setup.py` | Native setup workflow |
|
||||
| `core/` | Authentication, middleware, persistence, sessions, and platform primitives |
|
||||
| `routes/` | HTTP and API route handlers |
|
||||
| `src/` | Application services, orchestration, tools, providers, and runtime helpers |
|
||||
| `services/` | Domain-oriented service packages |
|
||||
| `mcp_servers/` | Built-in MCP server implementations |
|
||||
| `scripts/` | CLI tools, diagnostics, maintenance, and migration helpers |
|
||||
| `static/` | No-build browser frontend and bundled assets |
|
||||
| `tests/` | Automated test suite and supporting test infrastructure |
|
||||
|
||||
## Directory snapshot
|
||||
|
||||
| Directory | Tracked files | Tracked Python files | Direct subdirectories |
|
||||
|---|---:|---:|---|
|
||||
| `src/` | 143 | 143 | `agent_tools/`, `model_capability_readers/`, `search/`, `tools/` |
|
||||
| `routes/` | 73 | 73 | `admin_wipe/`, `cleanup/`, `compare/`, `contacts/`, `gallery/`, `history/`, `memory/`, `note/`, `research/` |
|
||||
| `core/` | 11 | 11 | None |
|
||||
| `services/` | 42 | 40 | `docs/`, `faces/`, `hwfit/`, `memory/`, `research/`, `search/`, `shell/`, `stt/`, `tts/`, `youtube/` |
|
||||
| `mcp_servers/` | 5 | 5 | None |
|
||||
| `scripts/` | 44 | 17 | `_completion/`, `_lib/`, `demo_email/` |
|
||||
| `static/js/` | 154 | 0 | `calendar/`, `color/`, `compare/`, `editor/`, `emailLibrary/`, `markdown/`, `model/`, `research/`, `util/` |
|
||||
| `tests/` | 768 | 758 | `cli/`, `helpers/`, `streaming/`, `tools/` |
|
||||
|
||||
> [!NOTE]
|
||||
> Counts in this table use `git ls-files`, so generated caches, virtual
|
||||
> environments, and other untracked local files are excluded.
|
||||
|
||||
## Largest backend modules
|
||||
|
||||
Large files are review signals, not proof that a module should be split.
|
||||
Coupling, ownership, import compatibility, tests, and runtime authority matter more
|
||||
than line count alone.
|
||||
|
||||
| Rank | File | Lines | Classes | Top-level functions | Review signal |
|
||||
|---:|---|---:|---:|---:|---|
|
||||
| 1 | `routes/email_routes.py` | 6032 | 1 | 58 | High |
|
||||
| 2 | `src/agent_loop.py` | 5248 | 0 | 63 | High |
|
||||
| 3 | `routes/cookbook_routes.py` | 4545 | 0 | 16 | High |
|
||||
| 4 | `mcp_servers/email_server.py` | 2920 | 0 | 77 | High |
|
||||
| 5 | `src/llm_core.py` | 2895 | 3 | 85 | High |
|
||||
| 6 | `src/builtin_actions.py` | 2845 | 2 | 27 | High |
|
||||
| 7 | `routes/model_routes.py` | 2743 | 0 | 65 | High |
|
||||
| 8 | `src/task_scheduler.py` | 2627 | 1 | 8 | Medium |
|
||||
| 9 | `core/database.py` | 2562 | 28 | 67 | High |
|
||||
| 10 | `routes/gallery/gallery_routes.py` | 2325 | 0 | 16 | Medium |
|
||||
| 11 | `routes/chat_routes.py` | 2063 | 0 | 18 | Medium |
|
||||
| 12 | `routes/shell_routes.py` | 1971 | 1 | 21 | Medium |
|
||||
| 13 | `src/visual_report.py` | 1933 | 0 | 11 | Medium |
|
||||
| 14 | `routes/email_helpers.py` | 1888 | 3 | 48 | Medium |
|
||||
| 15 | `routes/document_routes.py` | 1810 | 0 | 5 | Medium |
|
||||
| 16 | `src/tools/cookbook.py` | 1705 | 0 | 34 | Medium |
|
||||
| 17 | `routes/calendar_routes.py` | 1667 | 2 | 19 | Medium |
|
||||
| 18 | `routes/skills_routes.py` | 1662 | 3 | 19 | Medium |
|
||||
| 19 | `src/tool_schemas.py` | 1595 | 0 | 3 | Medium |
|
||||
| 20 | `routes/email_pollers.py` | 1551 | 0 | 23 | Medium |
|
||||
|
||||
The largest current backend concentrations include:
|
||||
|
||||
- email routing and helper logic;
|
||||
- agent-loop orchestration;
|
||||
- Cookbook lifecycle and serving logic;
|
||||
- provider and model routing;
|
||||
- task scheduling;
|
||||
- shared database models and persistence helpers.
|
||||
|
||||
These areas require focused ownership and compatibility analysis before structural
|
||||
changes are attempted.
|
||||
|
||||
## Largest frontend modules
|
||||
|
||||
| Rank | File | Lines |
|
||||
|---:|---|---:|
|
||||
| 1 | `static/style.css` | 41132 |
|
||||
| 2 | `static/js/document.js` | 11200 |
|
||||
| 3 | `static/js/emailLibrary.js` | 8505 |
|
||||
| 4 | `static/js/slashCommands.js` | 6520 |
|
||||
| 5 | `static/js/chat.js` | 6001 |
|
||||
| 6 | `static/js/settings.js` | 5819 |
|
||||
| 7 | `static/js/notes.js` | 5365 |
|
||||
| 8 | `static/app.js` | 4681 |
|
||||
| 9 | `static/js/cookbookRunning.js` | 4433 |
|
||||
| 10 | `static/js/galleryEditor.js` | 4386 |
|
||||
| 11 | `static/js/cookbookServe.js` | 4305 |
|
||||
| 12 | `static/js/calendar.js` | 3722 |
|
||||
| 13 | `static/js/cookbook.js` | 3677 |
|
||||
| 14 | `static/js/sessions.js` | 3665 |
|
||||
| 15 | `static/js/documentLibrary.js` | 3422 |
|
||||
| 16 | `static/js/tasks.js` | 3187 |
|
||||
| 17 | `static/js/admin.js` | 3144 |
|
||||
| 18 | `static/js/gallery.js` | 2958 |
|
||||
| 19 | `static/js/cookbook-hwfit.js` | 2826 |
|
||||
| 20 | `static/js/chatRenderer.js` | 2808 |
|
||||
|
||||
The browser frontend remains a no-build ES-module application. Its current source
|
||||
tree is authoritative; the maintained structural summary is available in
|
||||
[`static/js/MODULE_SUMMARY.md`](../static/js/MODULE_SUMMARY.md).
|
||||
|
||||
CSS modularization remains tracked separately in
|
||||
[#2617](https://github.com/odysseus-dev/odysseus/issues/2617).
|
||||
|
||||
## Major subsystem boundaries
|
||||
|
||||
| Subsystem | Primary implementation locations |
|
||||
|---|---|
|
||||
| Application startup | `app.py`, `src/app_initializer.py`, `core/` |
|
||||
| Authentication and sessions | `core/auth.py`, `core/middleware.py`, `core/session_manager.py`, `routes/auth_routes.py` |
|
||||
| Chat and streaming | `routes/chat_routes.py`, `routes/chat_helpers.py`, `src/chat_handler.py`, `src/chat_processor.py`, `src/llm_core.py` |
|
||||
| Agents and tools | `src/agent_loop.py`, `src/tool_execution.py`, `src/agent_tools/`, `src/tools/`, `src/tool_policy.py`, `src/tool_security.py` |
|
||||
| Models and providers | `routes/model_routes.py`, `src/model_discovery.py`, `src/model_capabilities.py`, `src/endpoint_resolver.py`, `src/llm_core.py` |
|
||||
| Cookbook and hardware fit | `routes/cookbook_routes.py`, `routes/cookbook_helpers.py`, `src/cookbook_serve_lifecycle.py`, `services/hwfit/` |
|
||||
| Search and research | `routes/search_routes.py`, `services/search/`, `routes/research/`, `services/research/`, `src/deep_research.py` |
|
||||
| Documents and retrieval | `routes/document_routes.py`, `src/document_processor.py`, `src/personal_docs.py`, `src/rag_manager.py`, `src/pdf_runtime.py` |
|
||||
| Memory and skills | `routes/memory/`, `services/memory/`, `routes/skills_routes.py` |
|
||||
| Email | `routes/email_routes.py`, `routes/email_helpers.py`, `routes/email_pollers.py`, `mcp_servers/email_server.py` |
|
||||
| Calendar, contacts, notes, and tasks | `routes/calendar_routes.py`, `routes/contacts/`, `routes/note/`, `routes/task_routes.py`, `src/task_scheduler.py` |
|
||||
| Media and speech | `routes/gallery/`, `routes/stt_routes.py`, `routes/tts_routes.py`, `services/stt/`, `services/tts/` |
|
||||
| Persistence and operations | `core/database.py`, `src/runtime_paths.py`, `src/bg_jobs.py`, `routes/backup_routes.py`, `routes/cleanup/` |
|
||||
|
||||
For a broader evidence map, see
|
||||
[`system-map.md`](./system-map.md).
|
||||
|
||||
## Refactor-sensitive areas
|
||||
|
||||
### Shared persistence
|
||||
|
||||
`core/database.py` is a central dependency containing models and shared persistence
|
||||
helpers. Changes can affect routes, services, background work, tests, migrations,
|
||||
and import compatibility.
|
||||
|
||||
A split should not begin from file size alone. It requires:
|
||||
|
||||
- an importer inventory;
|
||||
- model and helper ownership decisions;
|
||||
- migration compatibility checks;
|
||||
- stable re-export or migration strategy;
|
||||
- focused and full-suite validation.
|
||||
|
||||
### Agent orchestration
|
||||
|
||||
`src/agent_loop.py` coordinates model interaction, tool selection, policy decisions,
|
||||
multi-round execution, and background behaviour. Extraction work must preserve tool
|
||||
event semantics, policy enforcement, cancellation, and test patch points.
|
||||
|
||||
Historical agent-loop modularization discussion is tracked in
|
||||
[#3266](https://github.com/odysseus-dev/odysseus/issues/3266).
|
||||
|
||||
### Tool implementation boundaries
|
||||
|
||||
Tool implementation is no longer represented by one proposed future package alone.
|
||||
Current responsibilities are distributed across:
|
||||
|
||||
- `src/tool_execution.py`;
|
||||
- `src/tool_schemas.py`;
|
||||
- `src/tool_index.py`;
|
||||
- `src/tool_policy.py`;
|
||||
- `src/tool_security.py`;
|
||||
- `src/agent_tools/`;
|
||||
- `src/tools/`;
|
||||
- remaining compatibility surfaces such as `src/tool_implementations.py`.
|
||||
|
||||
Historical tool modularization work is tracked in
|
||||
[#3629](https://github.com/odysseus-dev/odysseus/issues/3629).
|
||||
|
||||
### Route ownership
|
||||
|
||||
`routes/` now contains both flat modules and domain packages. Existing package
|
||||
boundaries should be extended only through focused changes. Broad mechanical route
|
||||
movement would affect registration, imports, tests, monkeypatch targets, and
|
||||
compatibility paths.
|
||||
|
||||
### Frontend concentration
|
||||
|
||||
The no-build frontend contains several large JavaScript modules and one central CSS
|
||||
file. Refactors should preserve module load order, global compatibility exports,
|
||||
DOM contracts, deep-link handling, and browser behaviour.
|
||||
|
||||
## Non-implemented architecture options
|
||||
|
||||
> [!NOTE]
|
||||
> The paths below are historical or possible design directions. They do not describe
|
||||
> the current repository and are not approved implementation plans.
|
||||
|
||||
Earlier planning discussed:
|
||||
|
||||
- renaming `app.py` to `main.py`;
|
||||
- moving agent orchestration into a new `src/agent/` package;
|
||||
- introducing broad `src/domain/`, `src/infra/`, `src/api/`, or `src/pkg/` layers;
|
||||
- moving all routes into domain subpackages;
|
||||
- splitting database models into a new infrastructure hierarchy.
|
||||
|
||||
These options should be reconsidered against the current tree rather than copied
|
||||
forward as assumed targets.
|
||||
|
||||
## Refactor guardrails
|
||||
|
||||
- Keep structural changes behaviour-preserving.
|
||||
- Change one ownership boundary at a time.
|
||||
- Do not mix file movement with unrelated feature work.
|
||||
- Preserve existing import and monkeypatch paths where compatibility is required.
|
||||
- Identify focused tests before modifying high-authority modules.
|
||||
- Validate startup, imports, and affected runtime paths.
|
||||
- Avoid repository-wide package reorganizations without maintainer agreement.
|
||||
- Treat generated metrics as snapshots, not architectural decisions.
|
||||
|
||||
## Reproduce the snapshot
|
||||
|
||||
Run these commands from the repository root.
|
||||
|
||||
```bash
|
||||
# Tracked directory totals
|
||||
for dir in src routes core services mcp_servers scripts static/js tests; do
|
||||
files="$(git ls-files "$dir" | wc -l)"
|
||||
python_files="$(git ls-files "$dir" '*.py' | wc -l)"
|
||||
|
||||
printf '%-14s tracked=%-5s python=%-5s\n' \
|
||||
"$dir" \
|
||||
"$files" \
|
||||
"$python_files"
|
||||
done
|
||||
|
||||
# Largest tracked backend files
|
||||
git ls-files \
|
||||
'app.py' \
|
||||
'launcher.py' \
|
||||
'setup.py' \
|
||||
'core/*.py' \
|
||||
'core/**/*.py' \
|
||||
'routes/*.py' \
|
||||
'routes/**/*.py' \
|
||||
'services/*.py' \
|
||||
'services/**/*.py' \
|
||||
'src/*.py' \
|
||||
'src/**/*.py' \
|
||||
'mcp_servers/*.py' \
|
||||
'scripts/*.py' \
|
||||
'scripts/**/*.py' |
|
||||
xargs wc -l |
|
||||
sort -nr |
|
||||
head -31
|
||||
|
||||
# Largest tracked frontend source files
|
||||
git ls-files \
|
||||
'static/*.js' \
|
||||
'static/*.css' \
|
||||
'static/*.html' \
|
||||
'static/**/*.js' \
|
||||
'static/**/*.css' \
|
||||
'static/**/*.html' |
|
||||
grep -vE '\.min\.js$' |
|
||||
xargs wc -l |
|
||||
sort -nr |
|
||||
head -31
|
||||
```
|
||||
|
||||
## Validation for architecture changes
|
||||
|
||||
Use the smallest relevant checks first, then expand according to risk:
|
||||
|
||||
```bash
|
||||
python3 -m compileall -q app.py core routes services src
|
||||
venv/bin/python -m pytest tests/<focused-test-file>.py -q
|
||||
venv/bin/python -m pytest -q
|
||||
```
|
||||
|
||||
Startup, browser, Docker, and integration checks may also be required depending on
|
||||
the affected boundary.
|
||||
|
||||
## Related documentation
|
||||
|
||||
- [Documentation style](../docs/STYLE.md)
|
||||
- [Discovery maps](../discovery/README.md)
|
||||
- [Current system map](../discovery/system-map.md)
|
||||
- [Safety boundaries](../discovery/safety-boundaries.md)
|
||||
- [Frontend module summary](../static/js/MODULE_SUMMARY.md)
|
||||
- [Testing standard](../tests/TESTING_STANDARD.md)
|
||||
@@ -0,0 +1,142 @@
|
||||
# Safety boundaries
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This non-canonical discovery map records code-grounded safeguards, confirmed risks or gaps, and unverified behaviour. Broad authority does not by itself establish a vulnerability. Verify the cited source before relying on a finding. No destructive test, external connection, or real credential was used for this map.
|
||||
|
||||
## Navigate the boundaries
|
||||
|
||||
- [Shell and subprocess execution](#shell-and-subprocess-execution)
|
||||
- [Filesystem access and workspace confinement](#filesystem-access-and-workspace-confinement)
|
||||
- [Agent-controlled tool dispatch](#agent-controlled-tool-dispatch)
|
||||
- [MCP and external tool servers](#mcp-and-external-tool-servers)
|
||||
- [Outbound network requests and URL validation](#outbound-network-requests-and-url-validation)
|
||||
- [Secrets, credentials, and vault sessions](#secrets-credentials-and-vault-sessions)
|
||||
- [Authentication and privileged administration](#authentication-and-privileged-administration)
|
||||
- [Deletion, wipe, backup, and restore](#deletion-wipe-backup-and-restore)
|
||||
- [Background jobs and unattended task execution](#background-jobs-and-unattended-task-execution)
|
||||
|
||||
## Shell and subprocess execution
|
||||
|
||||
- **Boundary:** Shell routes, agent `bash` and `python` tools, local model serving, and detached background jobs.
|
||||
|
||||
- **Available authority:** Commands run as the application process user and can create child processes.
|
||||
|
||||
- **User-controlled inputs:** Direct shell requests, model-produced tool arguments, scheduled-task prompts, and model-serving configuration.
|
||||
|
||||
- **Current safeguards:** Agent dispatch applies owner/admin checks and tool policy; process helpers use timeouts or bounded background-job lifecycle where implemented.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with a confirmed gap: the agent shell starts in its workspace but is not sandboxed to it, and has no egress sandbox. This is documented in source and the threat model; it is not a newly demonstrated bypass.
|
||||
|
||||
- **Unverified behaviour:** Role-gate and disabled-tool outcomes, direct shell-route behaviour, and timeout, cancellation, and output handling for foreground and detached processes remain unverified.
|
||||
|
||||
## Filesystem access and workspace confinement
|
||||
|
||||
- **Boundary:** Agent read, write, patch, listing, glob, and grep tools.
|
||||
|
||||
- **Available authority:** Read and modify files within active workspace confinement or fallback allowlisted roots.
|
||||
|
||||
- **User-controlled inputs:** Tool paths, patches, file contents, search patterns, and workspace selection passed into the tool dispatcher.
|
||||
|
||||
- **Current safeguards:** [`src/tool_execution.py`](../src/tool_execution.py) resolves paths, blocks sensitive subpaths, applies allowlist containment, and tightens paths to the active workspace when one is bound. File tools use those resolvers.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with safeguards. The file-tool policy does not sandbox the shell; treating a workspace as a whole-process containment boundary would be incorrect.
|
||||
|
||||
- **Unverified behaviour:** Traversal, symlink, sensitive-name, absolute-path, and workspace-switch behaviour remains unverified.
|
||||
|
||||
## Agent-controlled tool dispatch
|
||||
|
||||
- **Boundary:** Model output becomes native or parsed tool calls and is dispatched by the agent loop.
|
||||
|
||||
- **Available authority:** The authority of every enabled tool, including privileged built-ins and external tools.
|
||||
|
||||
- **User-controlled inputs:** Chat content, attached/retrieved content that may influence the model, tool arguments, per-request tool selection, and policy toggles.
|
||||
|
||||
- **Current safeguards:** [`src/tool_security.py`](../src/tool_security.py) blocks protected tools for non-admin users and fails closed for malformed tool names; [`src/tool_policy.py`](../src/tool_policy.py) supports disabled and guide-only policy; prompt-security helpers label untrusted context.
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: aliases, legacy text tools, native function calls, and MCP-qualified names must all reach the same policy outcome. The code has specific alias handling for email/MCP names, which makes this a sensitive compatibility seam.
|
||||
|
||||
- **Unverified behaviour:** The current policy outcomes for owner role, request mode, disabled state, native versus parsed invocation, qualified aliases, and external-content entry points remain unverified.
|
||||
|
||||
## MCP and external tool servers
|
||||
|
||||
- **Boundary:** Configured MCP servers and their tools are exposed to the agent through the MCP manager and routes.
|
||||
|
||||
- **Available authority:** Depends on the server: external network access, local process access, messaging, or data mutation may be delegated outside the application.
|
||||
|
||||
- **User-controlled inputs:** Server configuration, remote OAuth completion, tool arguments, and model-selected MCP calls.
|
||||
|
||||
- **Current safeguards:** MCP routes are registered through [`routes/mcp_routes.py`](../routes/mcp_routes.py); MCP-qualified tools are denied to non-admin users by [`src/tool_security.py`](../src/tool_security.py). OAuth state and token persistence are handled in [`src/mcp_oauth.py`](../src/mcp_oauth.py).
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: an MCP server authority is broader than the application can infer from its tool name. This map does not establish a trust or approval model for server installation and individual tool invocation.
|
||||
|
||||
- **Unverified behaviour:** Server onboarding, credential storage, server-origin trust, OAuth callback deployment, tool disablement, and invocation audit behaviour remain unverified.
|
||||
|
||||
## Outbound network requests and URL validation
|
||||
|
||||
- **Boundary:** Search/content fetch, research, webhooks, skill import, provider endpoints, and other HTTP clients.
|
||||
|
||||
- **Available authority:** The application can make outbound requests from its network position.
|
||||
|
||||
- **User-controlled inputs:** Search/fetch URLs, imported skill URLs, webhook configuration, and some endpoint settings.
|
||||
|
||||
- **Current safeguards:** [`src/url_security.py`](../src/url_security.py) validates untrusted public HTTP URLs and fails closed on unsuitable schemes or private addresses. [`services/search/content.py`](../services/search/content.py) resolves and rejects non-public hosts, pins resolved addresses for fetches, caps bodies, and limits redirects.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional split: administrator-created model endpoints may target private providers, while untrusted URLs use public-address checks. That distinction is required for self-hosted deployments but needs explicit call-site review.
|
||||
|
||||
- **Unverified behaviour:** The URL-source classification for outbound clients and the current handling of redirects and DNS changes remain unverified.
|
||||
|
||||
## Secrets, credentials, and vault sessions
|
||||
|
||||
- **Boundary:** Application-managed encrypted secrets, API keys, provider credentials, and Bitwarden/Vaultwarden CLI sessions.
|
||||
|
||||
- **Available authority:** Credentials unlock remote providers and connected personal services.
|
||||
|
||||
- **User-controlled inputs:** Administrative configuration, login/unlock requests, imported settings, and agent vault tool arguments.
|
||||
|
||||
- **Current safeguards:** [`src/secret_storage.py`](../src/secret_storage.py) uses a locally stored Fernet key with restrictive permissions for supported database secrets. Vault routes require an administrator, avoid passing master passwords in command arguments, and set restrictive permissions on the vault-session file.
|
||||
|
||||
- **Confirmed risks or gaps:** Confirmed current boundary: vault session data is persisted through the vault path, not through [`src/secret_storage.py`](../src/secret_storage.py). This is an unresolved question about current security semantics, not a confirmed exposure.
|
||||
|
||||
- **Unverified behaviour:** Current encryption-at-rest, owner scope, rotation, lock/logout, backup/restore, and log/tool-result exposure behaviour remains unverified.
|
||||
|
||||
## Authentication and privileged administration
|
||||
|
||||
- **Boundary:** Session authentication, API tokens, privileged routes, and internal tool loopback.
|
||||
|
||||
- **Available authority:** Administrative identity can access execution, settings, integrations, data deletion, and secrets.
|
||||
|
||||
- **User-controlled inputs:** Login/signup data, session cookies, API tokens, authentication configuration, and requests to privileged routes.
|
||||
|
||||
- **Current safeguards:** [`core/auth.py`](../core/auth.py), [`core/middleware.py`](../core/middleware.py), and route-level checks establish identity and administrator gates. [`app.py`](../app.py) warns when localhost bypass is configured; [`SECURITY.md`](../SECURITY.md) documents deployment requirements.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with safeguards. Security depends on deployments keeping authentication enabled and internal services private; this map does not audit reverse-proxy or environment configuration.
|
||||
|
||||
- **Unverified behaviour:** Setup, anonymous, non-admin, admin, token, and internal-loopback behaviour, including privileged-route gate consistency, remains unverified.
|
||||
|
||||
## Deletion, wipe, backup, and restore
|
||||
|
||||
- **Boundary:** Administrative wipe, cleanup, backup import/export, and the backup restore command.
|
||||
|
||||
- **Available authority:** Delete or replace user data and credentials.
|
||||
|
||||
- **User-controlled inputs:** Administrative HTTP requests, cleanup choices, backup payloads, archive paths, and restore command options.
|
||||
|
||||
- **Current safeguards:** Administrative wipe routes use the administrative boundary. Cleanup exposes a preview route before mutation. The documented backup tool requires explicit restore confirmation, stages the old data directory, and validates archive members before extraction.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional destructive authority. Backup archives contain secrets by design, as documented in [`docs/backup-restore.md`](../docs/backup-restore.md); this is an operator confidentiality responsibility, not a code defect established here.
|
||||
|
||||
- **Unverified behaviour:** Role-gate, confirmation, archive-rejection, staged-recovery, and owner-isolation behaviour remains unverified. No destructive runtime test was performed.
|
||||
|
||||
## Background jobs and unattended task execution
|
||||
|
||||
- **Boundary:** Scheduled tasks, background-job monitor, startup tasks, and notification/delivery work that continue without an active browser request.
|
||||
|
||||
- **Available authority:** Scheduled agent work can obtain model access and, for eligible owners, shell and file tools; task output can interact with connected services.
|
||||
|
||||
- **User-controlled inputs:** Stored task prompt, schedule, model/crew selection, enabled-tool configuration, output target, and prior persisted state.
|
||||
|
||||
- **Current safeguards:** [`src/task_scheduler.py`](../src/task_scheduler.py) serializes execution, records task runs, associates work with an owner, and applies the agent owner-based tool gate. [`src/bg_jobs.py`](../src/bg_jobs.py) keeps bounded state and can terminate overlong subprocess jobs.
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: authority is inherited and exercised later, so changes to roles, task configuration, and disabled tools must be checked at execution time rather than assumed from task creation.
|
||||
|
||||
- **Unverified behaviour:** Creation, editing, role-change, scheduling, cancellation, restart-recovery, and execution behaviour remains unverified, including whether current policy is re-evaluated before privileged action.
|
||||
@@ -0,0 +1,139 @@
|
||||
# Current system map
|
||||
|
||||
> [!NOTE]
|
||||
> This non-canonical discovery map is an evidence guide, not an exhaustive feature catalog or runtime certification. Verify the cited source before relying on a finding. Each section records local implementation observations, evidence locations, confirmed current problems, and unresolved factual questions.
|
||||
|
||||
## Navigate the system
|
||||
|
||||
- [Startup and application composition](#startup-and-application-composition)
|
||||
- [Frontend shell and browser interaction](#frontend-shell-and-browser-interaction)
|
||||
- [Chat, sessions, and streaming](#chat-sessions-and-streaming)
|
||||
- [Agents, tools, and execution](#agents-tools-and-execution)
|
||||
- [Models, providers, and local serving](#models-providers-and-local-serving)
|
||||
- [Search and research](#search-and-research)
|
||||
- [Documents, retrieval, and personal knowledge](#documents-retrieval-and-personal-knowledge)
|
||||
- [Memory and skills](#memory-and-skills)
|
||||
- [Email, calendar, contacts, notes, and tasks](#email-calendar-contacts-notes-and-tasks)
|
||||
- [Media, speech, and image work](#media-speech-and-image-work)
|
||||
- [Authentication, secrets, and privileged administration](#authentication-secrets-and-privileged-administration)
|
||||
- [Persistence, background work, and operations](#persistence-background-work-and-operations)
|
||||
|
||||
## Startup and application composition
|
||||
|
||||
- **How it works:** [`app.py`](../app.py) creates the application, mounts static assets, constructs shared services, registers route factories, and owns lifespan startup and shutdown. [`src/app_initializer.py`](../src/app_initializer.py) prepares application state; [`core/`](../core/) provides persistence, authentication, middleware, sessions, and platform helpers.
|
||||
|
||||
- **Evidence locations:** [`app.py`](../app.py); [`src/app_initializer.py`](../src/app_initializer.py); [`core/database.py`](../core/database.py); [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/`](../routes/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which component currently owns startup and shutdown for each long-lived service?
|
||||
|
||||
## Frontend shell and browser interaction
|
||||
|
||||
- **How it works:** [`static/index.html`](../static/index.html) is served by the root and SPA deep-link routes in [`app.py`](../app.py); [`static/app.js`](../static/app.js), [`static/style.css`](../static/style.css), and [`static/js/`](../static/js/) implement the client surface.
|
||||
|
||||
- **Evidence locations:** [`static/index.html`](../static/index.html); [`static/app.js`](../static/app.js); [`static/js/`](../static/js/); [`static/style.css`](../static/style.css); [`app.py`](../app.py) deep-link handlers.
|
||||
|
||||
- **Known problems:** The `/backgrounds` route in [`app.py`](../app.py) calls `serve_html_with_nonce` for `static/backgrounds.html`, but that file is absent from [`static/`](../static/). This is a confirmed broken prototype route, not evidence about the rest of the frontend.
|
||||
|
||||
- **Open question:** Is `/backgrounds` currently an intentionally supported route or an obsolete prototype?
|
||||
|
||||
## Chat, sessions, and streaming
|
||||
|
||||
- **How it works:** [`routes/chat_routes.py`](../routes/chat_routes.py) and [`routes/chat_helpers.py`](../routes/chat_helpers.py) coordinate requests, session state, and SSE delivery. [`src/chat_handler.py`](../src/chat_handler.py), [`src/chat_processor.py`](../src/chat_processor.py), [`src/llm_core.py`](../src/llm_core.py), and [`src/session_actions.py`](../src/session_actions.py) provide message preparation, provider interaction, and session operations.
|
||||
|
||||
- **Evidence locations:** [`routes/chat_routes.py`](../routes/chat_routes.py); [`routes/chat_helpers.py`](../routes/chat_helpers.py); [`routes/session_routes.py`](../routes/session_routes.py); [`src/chat_handler.py`](../src/chat_handler.py); [`src/chat_processor.py`](../src/chat_processor.py); [`src/llm_core.py`](../src/llm_core.py); [`core/session_manager.py`](../core/session_manager.py).
|
||||
|
||||
- **Known problems:** [`src/agent_loop.py`](../src/agent_loop.py) annotates `_resolved_tool_event_name` with `Any` but imports no `Any` and does not enable postponed annotation evaluation. Python evaluates that annotation while importing the module, so this is an import-time defect at the checked baseline.
|
||||
|
||||
- **Open question:** No end-to-end provider or browser streaming run was performed for this map.
|
||||
|
||||
## Agents, tools, and execution
|
||||
|
||||
- **How it works:** [`src/agent_loop.py`](../src/agent_loop.py) drives multi-round tool use. [`src/tool_execution.py`](../src/tool_execution.py) dispatches calls and binds workspace context. [`src/agent_tools/`](../src/agent_tools/) contains individual implementations; [`src/tool_security.py`](../src/tool_security.py) and [`src/tool_policy.py`](../src/tool_policy.py) apply role and request policies. Long-running command work is represented by [`src/bg_jobs.py`](../src/bg_jobs.py).
|
||||
|
||||
- **Evidence locations:** [`src/agent_loop.py`](../src/agent_loop.py); [`src/tool_execution.py`](../src/tool_execution.py); [`src/agent_tools/`](../src/agent_tools/); [`src/tool_security.py`](../src/tool_security.py); [`src/tool_policy.py`](../src/tool_policy.py); [`src/tool_schemas.py`](../src/tool_schemas.py); [`src/bg_jobs.py`](../src/bg_jobs.py).
|
||||
|
||||
- **Known problems:** The import-time annotation defect above blocks the main agent/tool path. The shell is intentionally not a filesystem or network sandbox; that is an authority boundary, not by itself a vulnerability claim.
|
||||
|
||||
- **Open question:** Which native, legacy, and MCP-qualified invocation paths reach each policy gate?
|
||||
|
||||
## Models, providers, and local serving
|
||||
|
||||
- **How it works:** Model routes delegate to discovery, capabilities, endpoint resolution, and LLM core modules. Cookbook routes and hardware-fit services handle model lifecycle and local-serving support.
|
||||
|
||||
- **Evidence locations:** [`routes/model_routes.py`](../routes/model_routes.py); [`src/model_discovery.py`](../src/model_discovery.py); [`src/model_capabilities.py`](../src/model_capabilities.py); [`src/endpoint_resolver.py`](../src/endpoint_resolver.py); [`src/llm_core.py`](../src/llm_core.py); [`routes/cookbook_routes.py`](../routes/cookbook_routes.py); [`src/cookbook_serve_lifecycle.py`](../src/cookbook_serve_lifecycle.py); [`services/hwfit/`](../services/hwfit/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which endpoint inputs are administrator-created and permitted to use private provider addresses?
|
||||
|
||||
## Search and research
|
||||
|
||||
- **How it works:** HTTP search routes use [`services/search/`](../services/search/); research is exposed through [`routes/research/`](../routes/research/) and implemented in [`services/research/`](../services/research/), [`src/deep_research.py`](../src/deep_research.py), and related helpers. [`src/search/`](../src/search/) remains an import-compatibility layer for callers not yet moved to `services.search`.
|
||||
|
||||
- **Evidence locations:** [`routes/search_routes.py`](../routes/search_routes.py); [`services/search/`](../services/search/); [`routes/research/research_routes.py`](../routes/research/research_routes.py); [`services/research/`](../services/research/); [`src/deep_research.py`](../src/deep_research.py); [`src/search/`](../src/search/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** No live provider request was made; provider configuration and network access remain unverified.
|
||||
|
||||
## Documents, retrieval, and personal knowledge
|
||||
|
||||
- **How it works:** Document routes coordinate upload handling, document processing, and editor actions. Personal-document and RAG modules use Chroma and embedding clients. PDF viewing uses the optional-dependency loader in [`src/pdf_runtime.py`](../src/pdf_runtime.py); form extraction and filling live separately in [`src/pdf_forms.py`](../src/pdf_forms.py) and [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
|
||||
|
||||
- **Evidence locations:** [`routes/document_routes.py`](../routes/document_routes.py); [`src/upload_handler.py`](../src/upload_handler.py); [`src/document_processor.py`](../src/document_processor.py); [`src/document_actions.py`](../src/document_actions.py); [`src/personal_docs.py`](../src/personal_docs.py); [`src/rag_manager.py`](../src/rag_manager.py); [`src/embeddings.py`](../src/embeddings.py); [`src/pdf_runtime.py`](../src/pdf_runtime.py); [`src/pdf_forms.py`](../src/pdf_forms.py); [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
|
||||
|
||||
- **Known problems:** PDF viewing/runtime loading and PDF form processing are separate implementations. That separation is confirmed and intentional in the source; it is not a defect without a reported behavioural failure.
|
||||
|
||||
- **Open question:** Optional PDF dependencies and representative uploaded documents were not exercised.
|
||||
|
||||
## Memory and skills
|
||||
|
||||
- **How it works:** Memory routes use [`services/memory/`](../services/memory/) and vector helpers. Skills are exposed through [`routes/skills_routes.py`](../routes/skills_routes.py), stored and managed in [`services/memory/skills.py`](../services/memory/skills.py), and may be imported through [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
|
||||
|
||||
- **Evidence locations:** [`routes/memory/memory_routes.py`](../routes/memory/memory_routes.py); [`services/memory/`](../services/memory/); [`src/memory.py`](../src/memory.py); [`src/memory_vector.py`](../src/memory_vector.py); [`routes/skills_routes.py`](../routes/skills_routes.py); [`services/memory/skills.py`](../services/memory/skills.py); [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which imported skill content can reach execution-capable paths, and which validation occurs before that point?
|
||||
|
||||
## Email, calendar, contacts, notes, and tasks
|
||||
|
||||
- **How it works:** Dedicated route modules own email, CalDAV calendar, CardDAV contacts, notes, and tasks. Supporting modules include email helpers and pollers, CalDAV sync and writeback, and the task scheduler.
|
||||
|
||||
- **Evidence locations:** [`routes/email_routes.py`](../routes/email_routes.py); [`routes/calendar_routes.py`](../routes/calendar_routes.py); [`routes/contacts/contacts_routes.py`](../routes/contacts/contacts_routes.py); [`routes/note/note_routes.py`](../routes/note/note_routes.py); [`routes/task_routes.py`](../routes/task_routes.py); [`routes/assistant_routes.py`](../routes/assistant_routes.py); [`src/caldav_sync.py`](../src/caldav_sync.py); [`src/caldav_writeback.py`](../src/caldav_writeback.py); [`src/task_scheduler.py`](../src/task_scheduler.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** External account behaviour, writeback, and delivery require controlled credentials and are not runtime-validated here.
|
||||
|
||||
## Media, speech, and image work
|
||||
|
||||
- **How it works:** Gallery and image routes coordinate media features. Service modules own speech and media integrations; [`src/generated_images.py`](../src/generated_images.py) and [`src/visual_report.py`](../src/visual_report.py) support artifact handling and presentation.
|
||||
|
||||
- **Evidence locations:** [`routes/gallery/gallery_routes.py`](../routes/gallery/gallery_routes.py); [`routes/stt_routes.py`](../routes/stt_routes.py); [`routes/tts_routes.py`](../routes/tts_routes.py); [`src/generated_images.py`](../src/generated_images.py); [`services/stt/`](../services/stt/); [`services/tts/`](../services/tts/); [`services/faces/`](../services/faces/); [`src/visual_report.py`](../src/visual_report.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Hardware- and provider-dependent media workflows were not exercised.
|
||||
|
||||
## Authentication, secrets, and privileged administration
|
||||
|
||||
- **How it works:** [`core/auth.py`](../core/auth.py) and [`core/middleware.py`](../core/middleware.py) provide identity and request gates. [`src/secret_storage.py`](../src/secret_storage.py) encrypts application-managed database secrets with a local Fernet key. Vault handling is separate: [`routes/vault_routes.py`](../routes/vault_routes.py) and [`src/tools/vault.py`](../src/tools/vault.py) invoke the Bitwarden CLI and persist its session data in the application data area.
|
||||
|
||||
- **Evidence locations:** [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/auth_routes.py`](../routes/auth_routes.py); [`routes/api_token_routes.py`](../routes/api_token_routes.py); [`src/secret_storage.py`](../src/secret_storage.py); [`routes/vault_routes.py`](../routes/vault_routes.py); [`src/tools/vault.py`](../src/tools/vault.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py).
|
||||
|
||||
- **Known problems:** Vault-command handling and local application secret storage are distinct paths with different storage mechanisms. This is a source-confirmed boundary, not evidence that either path is compromised.
|
||||
|
||||
- **Open question:** What are the current confidentiality, ownership, rotation, and backup semantics for vault session data?
|
||||
|
||||
## Persistence, background work, and operations
|
||||
|
||||
- **How it works:** SQLite models and persistence are centred in [`core/database.py`](../core/database.py); managers use application data paths. The scheduler and background-job monitor can continue work outside a live browser request. Operational routes cover cleanup, backup, and administrative wipe; the repository also provides a backup script and user documentation.
|
||||
|
||||
- **Evidence locations:** [`core/database.py`](../core/database.py); [`src/runtime_paths.py`](../src/runtime_paths.py); [`src/task_scheduler.py`](../src/task_scheduler.py); [`src/bg_jobs.py`](../src/bg_jobs.py); [`src/bg_monitor.py`](../src/bg_monitor.py); [`routes/backup_routes.py`](../routes/backup_routes.py); [`routes/cleanup/cleanup_routes.py`](../routes/cleanup/cleanup_routes.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py); [`scripts/odysseus-backup`](../scripts/odysseus-backup); [`docs/backup-restore.md`](../docs/backup-restore.md).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** What current behaviour applies to background execution, cancellation, retries, and authority inheritance?
|
||||
@@ -67,7 +67,6 @@ services:
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
|
||||
@@ -66,7 +66,6 @@ services:
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
|
||||
@@ -55,7 +55,6 @@ services:
|
||||
- ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES=${ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES:-26214400}
|
||||
- ODYSSEUS_STT_MAX_AUDIO_BYTES=${ODYSSEUS_STT_MAX_AUDIO_BYTES:-26214400}
|
||||
- ODYSSEUS_ICS_MAX_BYTES=${ODYSSEUS_ICS_MAX_BYTES:-10485760}
|
||||
- ODYSSEUS_TTS_CACHE_MAX_BYTES=${ODYSSEUS_TTS_CACHE_MAX_BYTES}
|
||||
- DATA_BRAVE_API_KEY=${DATA_BRAVE_API_KEY:-}
|
||||
- GOOGLE_API_KEY=${GOOGLE_API_KEY:-}
|
||||
- GOOGLE_PSE_CX=${GOOGLE_PSE_CX:-}
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
# Architecture
|
||||
|
||||
> [!NOTE]
|
||||
> This document is the proposed canonical destination for stable high-level
|
||||
> architecture facts. It remains subject to maintainer review. Source code,
|
||||
> tests, and configuration remain authoritative for implementation-sensitive
|
||||
> behaviour.
|
||||
|
||||
## Purpose
|
||||
|
||||
This document identifies the stable runtime boundaries and primary ownership
|
||||
locations used to navigate and extend Odysseus.
|
||||
|
||||
It intentionally excludes generated metrics, file-size rankings, refactor
|
||||
priorities, unresolved investigation findings, and proposed package layouts.
|
||||
|
||||
## Runtime structure
|
||||
|
||||
| Area | Responsibility |
|
||||
|---|---|
|
||||
| `app.py` | FastAPI application composition and primary application entry point |
|
||||
| `launcher.py` | Application launch support |
|
||||
| `setup.py` | Native setup workflow |
|
||||
| `core/` | Authentication, middleware, persistence, sessions, and platform primitives |
|
||||
| `routes/` | HTTP and API route handlers |
|
||||
| `src/` | Application orchestration, tools, providers, and runtime helpers |
|
||||
| `services/` | Domain-oriented service implementations |
|
||||
| `mcp_servers/` | Built-in MCP server implementations |
|
||||
| `scripts/` | CLI tools, diagnostics, maintenance, and migration helpers |
|
||||
| `static/` | No-build browser frontend and bundled assets |
|
||||
| `tests/` | Automated tests and supporting test infrastructure |
|
||||
|
||||
## Subsystem boundaries
|
||||
|
||||
| Subsystem | Primary implementation locations |
|
||||
|---|---|
|
||||
| Application startup | `app.py`, `src/app_initializer.py`, `core/` |
|
||||
| Authentication and sessions | `core/auth.py`, `core/middleware.py`, `core/session_manager.py`, `routes/auth_routes.py` |
|
||||
| Chat and streaming | `routes/chat_routes.py`, `routes/chat_helpers.py`, `src/chat_handler.py`, `src/chat_processor.py`, `src/llm_core.py` |
|
||||
| Agents and tools | `src/agent_loop.py`, `src/tool_execution.py`, `src/agent_tools/`, `src/tools/`, `src/tool_policy.py`, `src/tool_security.py` |
|
||||
| Models and providers | `routes/model_routes.py`, `src/model_discovery.py`, `src/model_capabilities.py`, `src/endpoint_resolver.py`, `src/llm_core.py` |
|
||||
| Cookbook and hardware fit | `routes/cookbook_routes.py`, `routes/cookbook_helpers.py`, `src/cookbook_serve_lifecycle.py`, `services/hwfit/` |
|
||||
| Search and research | `routes/search_routes.py`, `services/search/`, `routes/research/`, `services/research/`, `src/deep_research.py` |
|
||||
| Documents and retrieval | `routes/document_routes.py`, `src/document_processor.py`, `src/personal_docs.py`, `src/rag_manager.py`, `src/pdf_runtime.py` |
|
||||
| Memory and skills | `routes/memory/`, `services/memory/`, `routes/skills_routes.py` |
|
||||
| Email | `routes/email_routes.py`, `routes/email_helpers.py`, `routes/email_pollers.py`, `mcp_servers/email_server.py` |
|
||||
| Calendar, contacts, notes, and tasks | `routes/calendar_routes.py`, `routes/contacts/`, `routes/note/`, `routes/task_routes.py`, `src/task_scheduler.py` |
|
||||
| Media and speech | `routes/gallery/`, `routes/stt_routes.py`, `routes/tts_routes.py`, `services/stt/`, `services/tts/` |
|
||||
| Persistence and operations | `core/database.py`, `src/runtime_paths.py`, `src/bg_jobs.py`, `routes/backup_routes.py`, `routes/cleanup/` |
|
||||
|
||||
## Architectural constraints
|
||||
|
||||
- Preserve established import and compatibility paths unless a focused change
|
||||
explicitly migrates them.
|
||||
- Keep HTTP concerns in route modules and reusable domain behaviour in runtime
|
||||
or service modules.
|
||||
- Treat shared persistence, agent orchestration, tool execution, and application
|
||||
startup as high-authority boundaries.
|
||||
- Change one ownership boundary at a time.
|
||||
- Do not mix structural movement with unrelated feature behaviour.
|
||||
- Validate affected imports, startup paths, compatibility surfaces, and tests.
|
||||
|
||||
## Frontend
|
||||
|
||||
The browser frontend is a no-build ES-module application under `static/`.
|
||||
|
||||
Its maintained module-level structure is documented in
|
||||
[`static/js/MODULE_SUMMARY.md`](../static/js/MODULE_SUMMARY.md).
|
||||
|
||||
## Investigation and snapshots
|
||||
|
||||
Non-canonical investigation material is maintained under [`discovery/`](../discovery/).
|
||||
|
||||
The following documents may contain dated observations, metrics, unresolved
|
||||
questions, or historical planning and must not be treated as specifications:
|
||||
|
||||
- [`discovery/system-map.md`](../discovery/system-map.md)
|
||||
- [`discovery/architecture-runtime-inventory.md`](../discovery/architecture-runtime-inventory.md)
|
||||
|
||||
## Documentation authority
|
||||
|
||||
- Code, tests, and configuration define implemented behaviour.
|
||||
- Mature specifications define accepted subsystem behaviour where they exist.
|
||||
- Following maintainer acceptance, this document will define the high-level
|
||||
architecture map.
|
||||
- Discovery documents preserve evidence and uncertainty but remain
|
||||
non-canonical.
|
||||
@@ -0,0 +1,72 @@
|
||||
# Documentation style
|
||||
|
||||
This guide defines the shared structure and writing conventions for Odysseus documentation.
|
||||
|
||||
## Principles
|
||||
|
||||
- Write for a clear audience and purpose.
|
||||
- State whether a document is canonical, informational, a snapshot, or planning material.
|
||||
- Prefer current behaviour over historical explanation.
|
||||
- Link to source files, tests, issues, or other documentation when useful.
|
||||
- Separate verified behaviour from assumptions, open questions, and future work.
|
||||
- Keep headings descriptive and consistent.
|
||||
- Use Markdown callouts where status or risk must be visible.
|
||||
- Do not use emojis.
|
||||
|
||||
## Document status
|
||||
|
||||
Use a status callout near the top when the document is not normal canonical guidance.
|
||||
|
||||
### Canonical documentation
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This document describes accepted current behaviour. Verify implementation-sensitive details against the current code and tests.
|
||||
|
||||
### Discovery material
|
||||
|
||||
> [!NOTE]
|
||||
> This is non-canonical discovery material. It records code-grounded observations and open questions.
|
||||
|
||||
### Snapshot or inventory
|
||||
|
||||
> [!WARNING]
|
||||
> This document is a dated snapshot. Counts, paths, and implementation details may drift as the codebase changes.
|
||||
|
||||
### Planning material
|
||||
|
||||
> [!NOTE]
|
||||
> This document records planning context. It does not define current runtime behaviour or guarantee future implementation.
|
||||
|
||||
## Recommended structure
|
||||
|
||||
Use the following sections where relevant:
|
||||
|
||||
1. Title
|
||||
2. Purpose or status callout
|
||||
3. Scope
|
||||
4. Current behaviour or guidance
|
||||
5. Safety, limitations, or known gaps
|
||||
6. Validation or evidence
|
||||
7. Related documentation
|
||||
|
||||
Not every document needs every section.
|
||||
|
||||
## Writing style
|
||||
|
||||
- Use concise sentences.
|
||||
- Prefer direct language.
|
||||
- Avoid jokes, filler, and informal warnings.
|
||||
- Avoid repeating the same guidance across several files.
|
||||
- Link to the owning document instead of duplicating large sections.
|
||||
- Use lists for procedures, requirements, and comparisons.
|
||||
- Use tables only when they improve scanning.
|
||||
- Use fenced code blocks with an appropriate language identifier.
|
||||
- Use relative repository links for internal files.
|
||||
|
||||
## Authority
|
||||
|
||||
The current code, tests, and configuration are the source of truth for implemented behaviour.
|
||||
|
||||
Canonical documentation describes accepted behaviour and supported workflows.
|
||||
|
||||
Discovery, inventory, and planning documents must identify themselves explicitly and must not silently become behavioural specifications.
|
||||
@@ -181,8 +181,8 @@ Dirty, blocked, conflicting, and unknown merge states are shown as risk/caution
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
python3 -m py_compile scripts/pr_blocker_audit.py tests/test_pr_blocker_audit.py
|
||||
python3 -m pytest tests/test_pr_blocker_audit.py -q
|
||||
venv/bin/python -m py_compile scripts/pr_blocker_audit.py tests/test_pr_blocker_audit.py
|
||||
venv/bin/python -m pytest tests/test_pr_blocker_audit.py -q
|
||||
python3 scripts/pr_blocker_audit.py --help
|
||||
git diff --check
|
||||
```
|
||||
|
||||
+35
-16
@@ -1,6 +1,28 @@
|
||||
# Odysseus Setup Guide
|
||||
|
||||
This page keeps the detailed install, deployment, troubleshooting, and configuration notes out of the front README.
|
||||
This guide covers installation, deployment, troubleshooting, and configuration.
|
||||
For a minimal Docker installation, start with the
|
||||
[repository README](../README.md#quick-start).
|
||||
|
||||
## On this page
|
||||
|
||||
- [Quick Start](#quick-start)
|
||||
- [Docker](#docker-recommended)
|
||||
- [Native Linux and macOS](#native-linux--macos)
|
||||
- [Apple Silicon](#apple-silicon)
|
||||
- [Native Windows](#native-windows)
|
||||
- [Troubleshooting and advanced setup](#troubleshooting--advanced-setup)
|
||||
- [Security notes](#security-notes)
|
||||
- [Configuration](#configuration)
|
||||
- [Architecture](#architecture)
|
||||
- [Data and backups](#data)
|
||||
|
||||
Related guidance:
|
||||
|
||||
- [Security policy](../SECURITY.md)
|
||||
- [Architecture overview](ARCHITECTURE.md)
|
||||
- [Backup and restore guide](backup-restore.md)
|
||||
- [Contributing guide](../CONTRIBUTING.md)
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -15,8 +37,8 @@ On first setup, Odysseus creates an admin account (`admin` unless
|
||||
For Docker installs, the same line is in `docker compose logs odysseus`.
|
||||
Use that for the first login, then change it in **Settings**.
|
||||
|
||||
Contributing? See [CONTRIBUTING.md](../CONTRIBUTING.md) for setup, testing, and
|
||||
pull request guidelines.
|
||||
Contributing? See the [contributing guide](../CONTRIBUTING.md) for development
|
||||
setup, testing, and pull request guidelines.
|
||||
|
||||
### Docker (recommended)
|
||||
```bash
|
||||
@@ -441,8 +463,8 @@ uv pip sync requirements.lock # reproduce it exactly la
|
||||
### Outlook / Office 365 email
|
||||
Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
|
||||
and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
|
||||
passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
|
||||
current limitation and the planned integration direction.
|
||||
passwords will fail. See the [Outlook and Microsoft 365 email guide](email-outlook.md)
|
||||
for the current limitation and planned integration direction.
|
||||
|
||||
## Security Notes
|
||||
Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
|
||||
@@ -530,19 +552,16 @@ npx -y @playwright/mcp@latest --version
|
||||
That installs `@playwright/mcp` plus Playwright (~300MB total). Restart Odysseus and the server will register at startup.
|
||||
|
||||
## Architecture
|
||||
```
|
||||
app.py # FastAPI entry point
|
||||
core/ auth, database, middleware, constants
|
||||
src/ llm_core, agent_loop, agent_tools, chat_processor, search/
|
||||
routes/ chat, session, document, memory, model … endpoints
|
||||
services/ docs, memory, search, hwfit (Cookbook) …
|
||||
static/ index.html + app.js + style.css + js/ (modular front-end)
|
||||
docs/ landing page (index.html) + preview clips
|
||||
```
|
||||
|
||||
For stable high-level runtime structure, subsystem boundaries, and documentation
|
||||
authority, see the [architecture overview](ARCHITECTURE.md).
|
||||
|
||||
Source code, tests, and configuration remain authoritative for
|
||||
implementation-sensitive behaviour.
|
||||
|
||||
## Data
|
||||
All user data lives in `data/` (gitignored): `app.db` (sessions, messages, documents),
|
||||
`memory.json`, `presets.json`, `uploads/`, `personal_docs/`, `chroma/`, `settings.json`.
|
||||
|
||||
To back up or restore everything in `data/`, see the
|
||||
[Backup & Restore guide](backup-restore.md).
|
||||
To protect or recover this data, follow the
|
||||
[backup and restore guide](backup-restore.md).
|
||||
|
||||
+1
-4
@@ -38,10 +38,7 @@ python-dateutil
|
||||
caldav
|
||||
cryptography
|
||||
bcrypt
|
||||
# Built-in servers use the v1 low-level Server decorator API. MCP SDK v2 is a
|
||||
# breaking rewrite, so keep fresh installs on the maintained v1 line until the
|
||||
# servers are migrated together.
|
||||
mcp<2
|
||||
mcp
|
||||
pyotp
|
||||
qrcode[pil]
|
||||
croniter
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Search route domain package (slice 2j, #4082/#4071).
|
||||
|
||||
Contains search_routes.py, migrated from the flat routes/ directory.
|
||||
Backward-compat shim at routes/search_routes.py re-exports from here.
|
||||
"""
|
||||
@@ -1,111 +0,0 @@
|
||||
"""Search routes — /api/search/config GET, /api/search POST."""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Any
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
|
||||
import time
|
||||
|
||||
from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO
|
||||
from services.search.core import _call_provider
|
||||
from services.search.providers import _get_provider_key, _get_search_instance
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def _request_values(request: Request) -> Dict[str, Any]:
|
||||
"""Accept JSON, form data, or query params for search endpoints.
|
||||
|
||||
The browser UI posts FormData, while the agent's generic app_api tool
|
||||
posts JSON. FastAPI Form(...) rejects JSON with a 422 before our handler
|
||||
runs, which made the model think SearXNG was broken.
|
||||
"""
|
||||
values: Dict[str, Any] = dict(request.query_params)
|
||||
content_type = (request.headers.get("content-type") or "").lower()
|
||||
try:
|
||||
if "application/json" in content_type:
|
||||
body = await request.json()
|
||||
if isinstance(body, dict):
|
||||
values.update(body)
|
||||
else:
|
||||
form = await request.form()
|
||||
values.update(dict(form))
|
||||
except Exception:
|
||||
pass
|
||||
return values
|
||||
|
||||
|
||||
def setup_search_routes(config) -> APIRouter:
|
||||
router = APIRouter(tags=["search"])
|
||||
|
||||
@router.get("/api/search/config")
|
||||
async def get_search_settings() -> Dict[str, Any]:
|
||||
return get_search_config()
|
||||
|
||||
@router.post("/api/search")
|
||||
async def do_web_search(request: Request) -> Dict[str, Any]:
|
||||
"""Standalone web search — returns context string + source list.
|
||||
|
||||
Used by Compare mode to pre-search once and share results across panes.
|
||||
"""
|
||||
values = await _request_values(request)
|
||||
query = str(values.get("query") or values.get("q") or "").strip()
|
||||
if not query:
|
||||
return {"context": "", "sources": [], "error": "query is required"}
|
||||
time_filter = values.get("time_filter") or values.get("freshness")
|
||||
if time_filter is not None:
|
||||
time_filter = str(time_filter).strip() or None
|
||||
try:
|
||||
context, sources = comprehensive_web_search(
|
||||
query, return_sources=True, time_filter=time_filter,
|
||||
)
|
||||
return {"context": context, "sources": sources}
|
||||
except Exception as e:
|
||||
logger.error(f"Standalone web search failed: {e}")
|
||||
return {"context": "", "sources": [], "error": str(e)}
|
||||
|
||||
@router.get("/api/search/providers")
|
||||
async def list_search_providers():
|
||||
"""Return available search providers with config status."""
|
||||
providers = []
|
||||
for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items():
|
||||
if pid == "disabled":
|
||||
continue
|
||||
available = True
|
||||
if needs_key and not _get_provider_key(pid):
|
||||
available = False
|
||||
if needs_url and pid == "searxng" and not _get_search_instance():
|
||||
available = False
|
||||
providers.append({
|
||||
"id": pid,
|
||||
"label": label,
|
||||
"available": available,
|
||||
})
|
||||
return providers
|
||||
|
||||
@router.post("/api/search/query")
|
||||
async def search_with_provider(request: Request) -> Dict[str, Any]:
|
||||
"""Search using a specific provider. Used by compare search mode."""
|
||||
values = await _request_values(request)
|
||||
query = str(values.get("query") or values.get("q") or "").strip()
|
||||
provider = str(values.get("provider") or "").strip()
|
||||
try:
|
||||
count = int(values.get("count") or values.get("limit") or 10)
|
||||
except Exception:
|
||||
count = 10
|
||||
if not query:
|
||||
return {"results": [], "provider": provider, "error": "query is required"}
|
||||
if provider not in PROVIDER_INFO or provider == "disabled":
|
||||
return {"results": [], "provider": provider, "error": "Unknown provider"}
|
||||
t0 = time.time()
|
||||
try:
|
||||
results = _call_provider(provider, query, min(count, 20))
|
||||
elapsed = round(time.time() - t0, 2)
|
||||
return {"results": results, "provider": provider, "time": elapsed}
|
||||
except Exception as e:
|
||||
elapsed = round(time.time() - t0, 2)
|
||||
logger.error(f"Search provider {provider} failed: {e}")
|
||||
return {"results": [], "provider": provider, "time": elapsed, "error": str(e)}
|
||||
|
||||
return router
|
||||
+107
-9
@@ -1,13 +1,111 @@
|
||||
"""Backward-compat shim — canonical location is routes/search/search_routes.py.
|
||||
"""Search routes — /api/search/config GET, /api/search POST."""
|
||||
|
||||
This module is replaced in ``sys.modules`` by the canonical module object so
|
||||
that ``import routes.search_routes`` and ``from routes.search_routes import X``
|
||||
keep resolving to the canonical module. Keeps existing import paths working
|
||||
after slice 2j (#4082/#4071).
|
||||
"""
|
||||
import logging
|
||||
from typing import Dict, Any
|
||||
|
||||
import sys as _sys
|
||||
from fastapi import APIRouter, Request
|
||||
|
||||
from routes.search import search_routes as _canonical # noqa: F401
|
||||
import time
|
||||
|
||||
_sys.modules[__name__] = _canonical
|
||||
from services.search import get_search_config, comprehensive_web_search, PROVIDER_INFO
|
||||
from services.search.core import _call_provider
|
||||
from services.search.providers import _get_provider_key, _get_search_instance
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def _request_values(request: Request) -> Dict[str, Any]:
|
||||
"""Accept JSON, form data, or query params for search endpoints.
|
||||
|
||||
The browser UI posts FormData, while the agent's generic app_api tool
|
||||
posts JSON. FastAPI Form(...) rejects JSON with a 422 before our handler
|
||||
runs, which made the model think SearXNG was broken.
|
||||
"""
|
||||
values: Dict[str, Any] = dict(request.query_params)
|
||||
content_type = (request.headers.get("content-type") or "").lower()
|
||||
try:
|
||||
if "application/json" in content_type:
|
||||
body = await request.json()
|
||||
if isinstance(body, dict):
|
||||
values.update(body)
|
||||
else:
|
||||
form = await request.form()
|
||||
values.update(dict(form))
|
||||
except Exception:
|
||||
pass
|
||||
return values
|
||||
|
||||
|
||||
def setup_search_routes(config) -> APIRouter:
|
||||
router = APIRouter(tags=["search"])
|
||||
|
||||
@router.get("/api/search/config")
|
||||
async def get_search_settings() -> Dict[str, Any]:
|
||||
return get_search_config()
|
||||
|
||||
@router.post("/api/search")
|
||||
async def do_web_search(request: Request) -> Dict[str, Any]:
|
||||
"""Standalone web search — returns context string + source list.
|
||||
|
||||
Used by Compare mode to pre-search once and share results across panes.
|
||||
"""
|
||||
values = await _request_values(request)
|
||||
query = str(values.get("query") or values.get("q") or "").strip()
|
||||
if not query:
|
||||
return {"context": "", "sources": [], "error": "query is required"}
|
||||
time_filter = values.get("time_filter") or values.get("freshness")
|
||||
if time_filter is not None:
|
||||
time_filter = str(time_filter).strip() or None
|
||||
try:
|
||||
context, sources = comprehensive_web_search(
|
||||
query, return_sources=True, time_filter=time_filter,
|
||||
)
|
||||
return {"context": context, "sources": sources}
|
||||
except Exception as e:
|
||||
logger.error(f"Standalone web search failed: {e}")
|
||||
return {"context": "", "sources": [], "error": str(e)}
|
||||
|
||||
@router.get("/api/search/providers")
|
||||
async def list_search_providers():
|
||||
"""Return available search providers with config status."""
|
||||
providers = []
|
||||
for pid, (label, needs_key, needs_url) in PROVIDER_INFO.items():
|
||||
if pid == "disabled":
|
||||
continue
|
||||
available = True
|
||||
if needs_key and not _get_provider_key(pid):
|
||||
available = False
|
||||
if needs_url and pid == "searxng" and not _get_search_instance():
|
||||
available = False
|
||||
providers.append({
|
||||
"id": pid,
|
||||
"label": label,
|
||||
"available": available,
|
||||
})
|
||||
return providers
|
||||
|
||||
@router.post("/api/search/query")
|
||||
async def search_with_provider(request: Request) -> Dict[str, Any]:
|
||||
"""Search using a specific provider. Used by compare search mode."""
|
||||
values = await _request_values(request)
|
||||
query = str(values.get("query") or values.get("q") or "").strip()
|
||||
provider = str(values.get("provider") or "").strip()
|
||||
try:
|
||||
count = int(values.get("count") or values.get("limit") or 10)
|
||||
except Exception:
|
||||
count = 10
|
||||
if not query:
|
||||
return {"results": [], "provider": provider, "error": "query is required"}
|
||||
if provider not in PROVIDER_INFO or provider == "disabled":
|
||||
return {"results": [], "provider": provider, "error": "Unknown provider"}
|
||||
t0 = time.time()
|
||||
try:
|
||||
results = _call_provider(provider, query, min(count, 20))
|
||||
elapsed = round(time.time() - t0, 2)
|
||||
return {"results": results, "provider": provider, "time": elapsed}
|
||||
except Exception as e:
|
||||
elapsed = round(time.time() - t0, 2)
|
||||
logger.error(f"Search provider {provider} failed: {e}")
|
||||
return {"results": [], "provider": provider, "time": elapsed, "error": str(e)}
|
||||
|
||||
return router
|
||||
|
||||
@@ -1409,7 +1409,7 @@ def setup_skills_routes(skills_manager: SkillsManager) -> APIRouter:
|
||||
|
||||
# Prefer the configured DEFAULT (→ Utility) model — not the current chat
|
||||
# session's model. Fall back to the caller's session model only if unset.
|
||||
url, model, headers = resolve_endpoint("utility", owner=user)
|
||||
url, model, headers = resolve_endpoint("default", owner=user)
|
||||
if not url or not model:
|
||||
url = url or ((body.get("endpoint_url") or "").strip() or None)
|
||||
model = model or ((body.get("model") or "").strip() or None)
|
||||
|
||||
@@ -50,7 +50,7 @@ import json
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -441,4 +441,4 @@ class Skill:
|
||||
|
||||
|
||||
def _now_iso() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
return datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
"""Multi-provider TTS service — dispatches to local Kokoro, OpenAI-compatible API, or browser."""
|
||||
|
||||
import io
|
||||
import os
|
||||
import wave
|
||||
import logging
|
||||
import hashlib
|
||||
@@ -42,11 +41,6 @@ class TTSService:
|
||||
self.cache_dir = Path(cache_dir)
|
||||
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._kokoro = None # lazy-init
|
||||
|
||||
try:
|
||||
self.max_cache_bytes = int(os.getenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", 500 * 1024 * 1024))
|
||||
except ValueError:
|
||||
self.max_cache_bytes = 500 * 1024 * 1024
|
||||
|
||||
# ── Settings ──
|
||||
|
||||
@@ -95,53 +89,6 @@ class TTSService:
|
||||
ext = ".mp3" if (len(data) >= 3 and (data[:3] == b'ID3' or (data[0] == 0xff and (data[1] & 0xe0) == 0xe0))) else ".wav"
|
||||
(self.cache_dir / f"{key}{ext}").write_bytes(data)
|
||||
|
||||
self._enforce_cache_limit()
|
||||
|
||||
def _enforce_cache_limit(self):
|
||||
"""Evicts oldest files if the cache exceeds the configured byte limit."""
|
||||
if self.max_cache_bytes <= 0:
|
||||
return
|
||||
|
||||
try:
|
||||
files = []
|
||||
total_size = 0
|
||||
|
||||
# Safely scan files and sum sizes, ignoring files deleted mid-scan
|
||||
for f in self.cache_dir.iterdir():
|
||||
try:
|
||||
if f.is_file() and f.suffix.lower() in (".mp3", ".wav"):
|
||||
files.append(f)
|
||||
total_size += f.stat().st_size
|
||||
except OSError:
|
||||
continue
|
||||
|
||||
if total_size > self.max_cache_bytes:
|
||||
logger.info(
|
||||
f"TTS cache ({total_size} bytes) exceeded limit ({self.max_cache_bytes} bytes). Evicting oldest files."
|
||||
)
|
||||
|
||||
# Sort files by modification time (oldest first)
|
||||
try:
|
||||
files.sort(key=lambda f: f.stat().st_mtime)
|
||||
except OSError as e:
|
||||
logger.warning(f"Failed to sort cache files by mtime: {e}")
|
||||
|
||||
# Trim down to 80% of max capacity
|
||||
target_size = self.max_cache_bytes * 0.8
|
||||
|
||||
while files and total_size > target_size:
|
||||
f = files.pop(0)
|
||||
try:
|
||||
size = f.stat().st_size
|
||||
f.unlink()
|
||||
total_size -= size
|
||||
except OSError as e:
|
||||
logger.warning(f"Failed to evict cache file {f}: {e}")
|
||||
continue
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Error enforcing TTS cache limit: {e}", exc_info=True)
|
||||
|
||||
def clear_cache(self):
|
||||
count = 0
|
||||
for f in self.cache_dir.glob("*.*"):
|
||||
|
||||
@@ -1,412 +1,31 @@
|
||||
# Architecture Runtime Inventory
|
||||
|
||||
> **Purpose**: Phase 0 planning baseline for codebase readability improvements (#4071).
|
||||
> **Parent issue**: [#4082](https://github.com/odysseus-dev/odysseus/issues/4082)
|
||||
> **Last updated**: dev@b58af42 | 2026-06-16
|
||||
> **Status**: Draft — to be reviewed before follow-up slices open.
|
||||
> **Snapshot basis**: Importer / file / import-line counts are refreshed to `dev@b58af42` (2026-06-16) and are recomputable via the commands in §3.4. **Line counts** in §2.1 / §2.2 are a snapshot from an earlier baseline and drift as `dev` moves — recompute any of them with `wc -l <file>`. This inventory tracks structure and risk, not live metrics.
|
||||
|
||||
This document maps the current runtime module structure, identifies high-risk boundaries, and recommends safe first refactor slices. It does **not** move files, change imports, or alter runtime behavior.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current Structure Overview
|
||||
|
||||
### 1.1 Top-Level Layout
|
||||
|
||||
```
|
||||
odysseus/
|
||||
├── app.py # FastAPI app entrypoint (1,145 lines)
|
||||
├── conf/ # Configuration (config.py, settings.py, settings_scrub.py)
|
||||
├── src/ # 95 flat .py files + 2 subdirectories
|
||||
│ ├── agent_tools/ # Tool helpers: document, filesystem, subprocess, web
|
||||
│ └── search/ # Search subsystem
|
||||
├── routes/ # 54 flat .py files — HTTP route handlers
|
||||
├── core/ # 10 files — database models, auth, middleware, session
|
||||
├── mcp_servers/ # 5 files — MCP server implementations
|
||||
├── scripts/ # CLI tools and one-shot scripts
|
||||
├── static/ # Frontend HTML/CSS/JS
|
||||
├── tests/ # 583 test files (~54,800 lines)
|
||||
└── services/ # (exists as needed)
|
||||
```
|
||||
|
||||
### 1.2 Directory Flatness Metric
|
||||
|
||||
| Directory | Flat `.py` Files | Subdirectories | Concern |
|
||||
|-----------|-----------------|----------------|---------|
|
||||
| `src/` | **95** | 2 (`agent_tools/`, `search/`) | No domain grouping; 95 files in one directory |
|
||||
| `routes/` | **54** | 0 | All route handlers in one flat directory |
|
||||
| `core/` | 10 | 0 | Manageable, but `database.py` is oversized |
|
||||
|
||||
---
|
||||
|
||||
## 2. Largest Runtime Modules
|
||||
|
||||
### 2.1 Python Backend
|
||||
|
||||
| Rank | File | Lines | Classes | Functions | Risk |
|
||||
|------|------|-------|---------|-----------|------|
|
||||
| 1 | `src/tool_implementations.py` | **4,032** | 0 | ~48 | **HIGH** |
|
||||
| 2 | `routes/email_routes.py` | **3,245** | — | — | **MEDIUM** |
|
||||
| 3 | `routes/cookbook_routes.py` | **2,969** | — | — | **MEDIUM** |
|
||||
| 4 | `src/agent_loop.py` | **2,961** | 0 | ~24 | **HIGH** |
|
||||
| 5 | `src/task_scheduler.py` | **2,330** | — | 5 | MEDIUM |
|
||||
| 6 | `routes/model_routes.py` | **2,266** | — | — | MEDIUM |
|
||||
| 7 | `core/database.py` | **2,265** | 28 | ~59 helpers | **HIGH** |
|
||||
| 8 | `src/builtin_actions.py` | **2,262** | 2 | ~24 | MEDIUM |
|
||||
| 9 | `src/llm_core.py` | **2,164** | — | — | MEDIUM |
|
||||
| 10 | `mcp_servers/email_server.py` | 2,197 | — | — | LOW (separate process) |
|
||||
| 11 | `src/visual_report.py` | 1,918 | — | — | LOW |
|
||||
| 12 | `routes/gallery_routes.py` | 1,896 | — | — | LOW |
|
||||
| 13 | `src/ai_interaction.py` | 1,846 | — | — | MEDIUM |
|
||||
| 14 | `routes/document_routes.py` | 1,717 | — | — | LOW |
|
||||
| 15 | `routes/skills_routes.py` | 1,648 | — | — | LOW |
|
||||
|
||||
**Heuristic**: Files > 2,000 lines with 20+ public symbols and many importers are the highest-risk splits. Files 1,000–2,000 lines are medium-risk if tightly coupled.
|
||||
|
||||
### 2.2 Frontend
|
||||
|
||||
| File | Lines | Concern |
|
||||
|------|-------|---------|
|
||||
| `static/style.css` | **36,653** | Entire app CSS in one file (tracked separately in #2617) |
|
||||
| `static/js/document.js` | **9,776** | Single JS file for document functionality |
|
||||
| `static/js/slashCommands.js` | 6,498 | |
|
||||
| `static/js/settings.js` | 5,266 | |
|
||||
| `static/js/emailLibrary.js` | 5,217 | |
|
||||
| `static/js/notes.js` | 5,124 | |
|
||||
| `static/js/chat.js` | 4,985 | |
|
||||
| `static/app.js` | 4,090 | |
|
||||
|
||||
**Note**: Frontend modularization is tracked separately in #2617 (CSS) and is not the focus of this Phase 0 inventory. Frontend is listed here for completeness but follow-up slices should target Python backend boundaries first.
|
||||
|
||||
---
|
||||
|
||||
## 3. Import Dependency Graph
|
||||
|
||||
### 3.1 Who Depends on `core/database.py`
|
||||
|
||||
**102 files** import from `core.database` — this is the most depended-upon module:
|
||||
|
||||
- All route handlers (`routes/*.py`)
|
||||
- Most `src/*.py` files
|
||||
- `core/session_manager.py`, `core/auth.py`
|
||||
- Multiple test files
|
||||
|
||||
**Implication**: Any split of `core/database.py` is the highest-risk refactor. It should be tackled **last**, never first.
|
||||
|
||||
### 3.2 Who Depends on `src/tool_implementations.py`
|
||||
|
||||
**17 files** import from `src.tool_implementations`:
|
||||
- `src/agent_loop.py`, `src/builtin_actions.py`, `src/tool_index.py`
|
||||
- `src/task_scheduler.py`, `src/tool_policy.py`
|
||||
- Various tests
|
||||
|
||||
### 3.3 Who Depends on `src/agent_loop.py`
|
||||
|
||||
**22 files** import from `src.agent_loop`:
|
||||
|
||||
- `src/tool_policy.py`, `src/teacher_escalation.py`, `src/bg_monitor.py`
|
||||
- `src/task_scheduler.py`
|
||||
- Multiple test files
|
||||
|
||||
### 3.4 Cross-Layer Import Violations
|
||||
|
||||
**`src/` importing from `routes/`** (backwards dependency — domain logic depending on HTTP layer):
|
||||
|
||||
```
|
||||
src/tool_implementations.py ──→ routes/calendar_routes.py
|
||||
src/tool_implementations.py ──→ routes/cookbook_helpers.py
|
||||
src/tool_implementations.py ──→ routes/email_helpers.py
|
||||
src/tool_implementations.py ──→ routes/email_pollers.py
|
||||
src/tool_implementations.py ──→ routes/email_routes.py
|
||||
src/tool_implementations.py ──→ routes/model_routes.py
|
||||
src/tool_implementations.py ──→ routes/note_routes.py
|
||||
src/tool_implementations.py ──→ routes/prefs_routes.py
|
||||
```
|
||||
|
||||
> These are **runtime imports** (inside function bodies, not at module top), which mitigates circular import risk but indicates fuzzy layer boundaries. Function-level inline imports from the HTTP layer into business logic are a code smell.
|
||||
|
||||
**Import counts (top-level)**:
|
||||
| Direction | Count | Notes |
|
||||
|-----------|-------|-------|
|
||||
| `routes/` → `src/` | **374** | Expected: HTTP handlers call domain logic |
|
||||
| `routes/` → `core/` | **126** | Expected: handlers access DB models |
|
||||
| `src/` → `routes/` | **31** | **Unexpected**: domain logic reaching into HTTP layer (direct grep of import lines referencing `routes/`) |
|
||||
| `src/` → `core/` | **106** | Acceptable but could be reduced with a data-access layer |
|
||||
|
||||
> **How the metrics in this document are computed** — recompute against current `dev` before treating any count as authoritative (the tree drifts; these numbers are a snapshot, not a live value):
|
||||
> - `src/` flat `.py` files: `find src -maxdepth 1 -name '*.py' | wc -l`
|
||||
> - `tests/` test files: `find tests -name 'test_*.py' | wc -l`
|
||||
> - `core.database` importers: `grep -rlE '(from|import) +core\.database' --include='*.py' . | grep -v core/database.py | wc -l`
|
||||
> - `src.agent_loop` importers: `grep -rlE '(from|import) +src\.agent_loop' --include='*.py' . | grep -v src/agent_loop.py | wc -l`
|
||||
> - Cross-layer import lines: `grep -rhE '(from|import) +<pkg>' --include='*.py' <dir>/ | wc -l` (e.g. `(from|import) +routes` over `src/`)
|
||||
|
||||
---
|
||||
|
||||
## 4. Route Ownership Map
|
||||
|
||||
Routes can be grouped into logical feature domains. Current flat structure obscures these boundaries:
|
||||
|
||||
| Domain | Route Files | Total Lines | Review Complexity |
|
||||
|--------|-------------|-------------|-------------------|
|
||||
| **Email** | `email_routes.py`, `email_helpers.py`, `email_pollers.py` | 5,936 | HIGH — most complex domain |
|
||||
| **Chat / Agent** | `chat_routes.py`, `chat_helpers.py`, `shell_routes.py`, `codex_routes.py`, `skills_routes.py` | 6,365 | HIGH — core interaction surface |
|
||||
| **Cookbook** | `cookbook_routes.py`, `cookbook_helpers.py`, `cookbook_output.py` | 4,110 | MEDIUM |
|
||||
| **Model / LLM** | `model_routes.py`, `assistant_routes.py`, `copilot_routes.py` | 2,764 | MEDIUM |
|
||||
| **Calendar / Contacts** | `calendar_routes.py`, `contacts_routes.py` | 2,336 | MEDIUM |
|
||||
| **Documents** | `document_routes.py`, `document_helpers.py` | 1,954 | LOW |
|
||||
| **Auth** | `auth_routes.py`, `api_token_routes.py`, `device_flow.py` | 1,171 | LOW |
|
||||
| **Tasks** | `task_routes.py` (standalone) | 1,157 | LOW |
|
||||
| **Session** | `session_routes.py` (standalone) | 1,287 | LOW |
|
||||
| **Gallery** | `gallery_routes.py`, `gallery_helpers.py` | 1,896 | LOW |
|
||||
| **Memory** | `memory_routes.py` | — | LOW |
|
||||
| **Research** | `research_routes.py` | — | LOW |
|
||||
| **MCP** | `mcp_routes.py` | — | LOW |
|
||||
| **Notes** | `note_routes.py` | — | LOW |
|
||||
| **Other** | `prefs_routes.py`, `upload_routes.py`, `vault_routes.py`, `webhook_routes.py`, `workspace_routes.py`, `search_routes.py`, `history_routes.py`, `hwfit_routes.py`, `preset_routes.py`, `signature_routes.py`, `backup_routes.py`, `cleanup_routes.py`, `diagnostics_routes.py`, `embedding_routes.py`, `emoji_routes.py`, `font_routes.py`, `stt_routes.py`, `tts_routes.py`, `compare_routes.py`, `personal_routes.py`, `editor_draft_routes.py`, `admin_wipe_routes.py`, `chatgpt_subscription_routes.py` | 2,000+ | LOW individual, HIGH cumulative |
|
||||
|
||||
---
|
||||
|
||||
## 5. Tool Registry & Implementation Boundaries
|
||||
|
||||
### 5.1 Current Tool Architecture
|
||||
|
||||
| Component | File | Lines | Role |
|
||||
|-----------|------|-------|------|
|
||||
| Tool schemas | `src/tool_schemas.py` | 1,392 | JSON Schema tool definitions (Duck-TypedDict) |
|
||||
| Tool index | `src/tool_index.py` | 542 | RAG-based tool retrieval from ChromaDB |
|
||||
| Tool implementations | `src/tool_implementations.py` | 4,032 | 33 `do_*` functions — all tool execution logic |
|
||||
| Tool security | `src/tool_security.py` | — | Owner-scoped tool blocking |
|
||||
| Tool policy | `src/tool_policy.py` | — | Guide-only directive, plan-mode disabled tools |
|
||||
| Tool utils | `src/tool_utils.py` | — | Shared tool helpers |
|
||||
|
||||
### 5.2 Tool Implementation Categories
|
||||
|
||||
The 33 `do_*` functions in `tool_implementations.py` fall into natural domain groups — the basis for slice 1's split in §6.2:
|
||||
|
||||
| Category | `do_*` functions | Count |
|
||||
|----------|------------------|-------|
|
||||
| **System / config** | `do_manage_skills`, `do_manage_tasks`, `do_manage_endpoints`, `do_manage_mcp`, `do_manage_webhooks`, `do_manage_tokens`, `do_manage_settings`, `do_api_call`, `do_app_api` | 9 |
|
||||
| **Cookbook / model serving** | `do_download_model`, `do_serve_model`, `do_list_served_models`, `do_stop_served_model`, `do_tail_serve_output`, `do_list_downloads`, `do_cancel_download`, `do_search_hf_models`, `do_adopt_served_model`, `do_list_cookbook_servers`, `do_list_serve_presets`, `do_serve_preset`, `do_list_cached_models` | 13 |
|
||||
| **Notes** | `do_manage_notes` | 1 |
|
||||
| **Calendar** | `do_manage_calendar` | 1 |
|
||||
| **Search** | `do_search_chats` | 1 |
|
||||
| **Research** | `do_manage_research`, `do_trigger_research` | 2 |
|
||||
| **Contacts** | `do_resolve_contact`, `do_manage_contact` | 2 |
|
||||
| **Vault** | `do_vault_search`, `do_vault_get`, `do_vault_unlock` | 3 |
|
||||
| **Image** | `do_edit_image` | 1 |
|
||||
| | **Total** | **33** |
|
||||
|
||||
> Low-level tools (filesystem, subprocess, web fetch, document parsing) live in `src/agent_tools/`, **not** in `tool_implementations.py` — out of scope for this split.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risk Assessment & Candidate Slice Ranking
|
||||
|
||||
> **Candidate proposals, not a committed plan.** The rankings, package shapes (e.g. `src/pkg/`, `src/domain/`, `src/infra/`, `src/api/`), split ordering, and route-grouping strategy below are **options for maintainer discussion**. Per #4082/#4071, slice ownership and order are settled by maintainers before any follow-up PR. §1–§3 above are the factual current-state inventory.
|
||||
|
||||
### 6.1 Risk Scale
|
||||
|
||||
| Level | Criteria |
|
||||
|-------|----------|
|
||||
| **LOW** | File has ≤3 importers AND ≤500 lines, OR is a pure refactor with clear boundaries |
|
||||
| **MEDIUM** | File has 4–15 importers OR 500–1,500 lines |
|
||||
| **HIGH** | File has 16+ importers OR >2,000 lines, OR has cross-layer import violations |
|
||||
|
||||
### 6.2 Ranked Split Candidates
|
||||
|
||||
| Priority | Target | Risk | Rationale |
|
||||
|----------|--------|------|-----------|
|
||||
| **1** | `src/tool_implementations.py` → `src/tools/*.py` | **MEDIUM** | 4,032 lines → ~10 files by tool category. Already has natural boundaries. 17 importers, tracked in #3629. Use `__init__.py` shim to keep existing imports working. |
|
||||
| **2** | `routes/` → domain subdirectories (one domain per PR) | **MEDIUM** | 54 flat files. Done **one domain at a time** (e.g. a standalone PR for the email domain, then chat, …), not a broad reorganization — route modules carry helper imports, registration assumptions, and test import paths. |
|
||||
| **3** | `src/agent_loop.py` → `src/agent/loop.py` + submodules | **MEDIUM-HIGH** | 2,961 lines, 24 functions. Can extract prompt building, classification, verification, and runaway detection. Tracked in #3266. |
|
||||
| **4** | `src/` → `src/pkg/`, `src/domain/`, `src/infra/`, `src/api/` | **MEDIUM** | Structural reorganization. Split flat `src/` into layered packages. Must come after routes and tools are stable. |
|
||||
| **5** | `routes/email_*.py` consolidation | **LOW** | Already grouped by filename prefix. Low-risk cleanup within the email domain. |
|
||||
| **6** | `core/database.py` → `src/infra/database/models/*.py` | **HIGH** | 28 classes, 102 importers. Highest-risk split. Must be **last** in any sequence. Requires careful import shim strategy. |
|
||||
| **7** | Frontend CSS modularization | **MEDIUM** | 36,653 lines. Tracked in #2617. Separate timeline from backend work. |
|
||||
| **8** | Frontend JS modularization | **MEDIUM** | 9,776 lines in `document.js`. Introduce ES modules at minimum. |
|
||||
|
||||
### 6.3 Candidate First 3 Behavior-Preserving Slices
|
||||
|
||||
**Slice 1: Split `tool_implementations.py`** (Lowest-risk high-impact)
|
||||
|
||||
- Create `src/tools/` package with one file per tool category
|
||||
- Add `src/tools/__init__.py` re-exporting all symbols with current names
|
||||
- Update 17 importers to use new paths (can be deferred via shim)
|
||||
- Validation: `python -m pytest tests/ -x -q` + manual smoke test of tool execution
|
||||
- Reference: #3629
|
||||
|
||||
**Slice 2: Group `routes/` by domain** (one domain per PR, not a broad sweep)
|
||||
|
||||
Route modules carry helper imports, router registration assumptions, and test import paths, so this must be done **one domain at a time** rather than as a single reorganization PR. Example sequence (each its own PR):
|
||||
|
||||
- PR 2a: move the **email** domain (`email_routes.py`, `email_helpers.py`, `email_pollers.py`) → `routes/email/` + shim
|
||||
- PR 2b: move the **chat/agent** domain → `routes/chat/` + shim
|
||||
- PR 2c: move the **cookbook** domain → `routes/cookbook/` + shim
|
||||
- …and so on per domain from §4
|
||||
|
||||
Each PR: add `__init__.py` re-exporting old names, update `app.py` router imports, validation `python app.py` starts clean. **No behavior change** — pure file reorganization.
|
||||
|
||||
**Slice 3: Extract `agent_loop.py` submodules** (Improve reviewability)
|
||||
|
||||
- Move prompt assembly → `src/agent/prompt.py`
|
||||
- Move request classification → `src/agent/classifier.py`
|
||||
- Move sub-agent verification → `src/agent/verifier.py`
|
||||
- Move runaway detection → `src/agent/runaway.py`
|
||||
- Move context management → `src/agent/context.py`
|
||||
- Keep `src/agent/loop.py` as the main orchestration module
|
||||
- Validation: `python -m pytest tests/test_agent_loop.py tests/test_loop_breaker_runaway.py -v`
|
||||
|
||||
---
|
||||
|
||||
## 7. Safety Guardrails for Follow-Up Work
|
||||
|
||||
Per maintainer guidance in #4082 and #4071:
|
||||
|
||||
- [ ] **One domain/slice per PR** — never mix multiple reorganizations
|
||||
- [ ] **No behavior changes** mixed with file moves — pure reorganization only
|
||||
- [ ] **Keep compatibility shims** — `__init__.py` re-exports for all existing import paths
|
||||
- [ ] **Add or identify focused tests** before risky splits
|
||||
- [ ] **Do not start with `core/database.py`** or broad route movement unless this inventory shows a safe boundary
|
||||
- [ ] **Prefer small, reviewable slices** over large restructures
|
||||
- [ ] **No packaging/runtime/tooling migration** mixed into file moves
|
||||
- [ ] **No frontend framework migration** inside this stabilization lane
|
||||
- [ ] **Validate with `python -m compileall`** — every PR must pass CI checks
|
||||
- [ ] **Validate with `pytest`** — run the full test suite before opening each PR
|
||||
|
||||
---
|
||||
|
||||
## 8. Validation Commands
|
||||
|
||||
Each follow-up PR should be verifiable with these commands before submission:
|
||||
|
||||
```bash
|
||||
# Syntax check — must pass with zero errors
|
||||
python -m compileall src/ routes/ core/ conf/
|
||||
|
||||
# Full test suite — must match baseline pass rate
|
||||
python -m pytest tests/ -x -q
|
||||
|
||||
# Import shim verification — existing import paths must still work
|
||||
python -c "from src.tool_implementations import do_search_chats; print('OK')"
|
||||
|
||||
# App startup smoke test (if backend touched)
|
||||
timeout 5 python app.py 2>&1 | head -5 || true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Open Questions
|
||||
|
||||
1. Is `#2538` (specs ground truth) the canonical behavior map baseline, and should this inventory be kept in sync with those specs once merged?
|
||||
2. Should route grouping follow the domain map proposed here, or is there a different taxonomy preferred by maintainers?
|
||||
3. For the `tool_implementations.py` split (#3629), is the tool categorization in §5.2 acceptable, or should it follow a different grouping?
|
||||
4. Should compatibility shims (`__init__.py`) be temporary (removed in a follow-up wave) or permanent?
|
||||
5. Should an ADR (Architecture Decision Record) document be started to track decisions made during this process?
|
||||
|
||||
---
|
||||
|
||||
## 10. Future Direction (NOT current state)
|
||||
|
||||
The following are **future refactor targets** (candidate directions **pending maintainer agreement**, not committed), recorded here so this inventory does not imply they exist today. None of them are present in the current `dev` tree:
|
||||
|
||||
- `main.py` — proposed rename of the `app.py` entrypoint. Today the app boots via `app.py`.
|
||||
- `src/agent/` — proposed package to hold `agent_loop.py` submodules (prompt/classifier/verifier/runaway/context). Today `agent_loop.py` is a single flat file in `src/`.
|
||||
- `src/infra/`, `src/domain/`, `src/pkg/`, `src/api/` — proposed layered reorganization of the flat `src/` directory (slice 4 in §6).
|
||||
|
||||
These become real only when the corresponding slices land.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: File Listing
|
||||
|
||||
### `src/` (95 files — 61 shown; run `ls src/*.py` for the full list)
|
||||
|
||||
```
|
||||
agent_loop.py tool_implementations.py tool_schemas.py
|
||||
tool_index.py tool_security.py tool_policy.py
|
||||
tool_utils.py builtin_actions.py task_scheduler.py
|
||||
llm_core.py model_context.py model_discovery.py
|
||||
session_search.py context_budget.py context_compactor.py
|
||||
ai_interaction.py action_intents.py agent_runs.py
|
||||
app_helpers.py app_initializer.py config.py
|
||||
database.py memory.py memory_provider.py
|
||||
secret_storage.py prompt_security.py url_security.py
|
||||
url_safety.py rate_limiter.py cleanup_service.py
|
||||
readiness.py service_health.py exceptions.py
|
||||
request_models.py assistant_log.py bg_monitor.py
|
||||
builtin_mcp.py chat_helpers.py chroma_client.py
|
||||
document_processor.py embedding_lanes.py deep_research.py
|
||||
research_handler.py research_utils.py personal_docs.py
|
||||
rag_manager.py rag_singleton.py topic_analyzer.py
|
||||
visual_report.py youtube_handler.py pdf_forms.py
|
||||
pdf_form_doc.py pdf_runtime.py caldav_writeback.py
|
||||
email_thread_parser.py text_helpers.py user_time.py
|
||||
teacher_escalation.py cookbook_serve_lifecycle.py
|
||||
chatgpt_subscription.py mcp_manager.py
|
||||
```
|
||||
|
||||
### `routes/` (54 files)
|
||||
|
||||
```
|
||||
__init__.py _validators.py
|
||||
auth_routes.py api_token_routes.py device_flow.py
|
||||
chat_routes.py chat_helpers.py shell_routes.py
|
||||
codex_routes.py skills_routes.py
|
||||
email_routes.py email_helpers.py email_pollers.py
|
||||
cookbook_routes.py cookbook_helpers.py cookbook_output.py
|
||||
model_routes.py assistant_routes.py copilot_routes.py
|
||||
calendar_routes.py contacts_routes.py
|
||||
document_routes.py document_helpers.py
|
||||
gallery_routes.py gallery_helpers.py
|
||||
task_routes.py session_routes.py
|
||||
note_routes.py memory_routes.py research_routes.py
|
||||
mcp_routes.py search_routes.py history_routes.py
|
||||
webhook_routes.py workspace_routes.py upload_routes.py
|
||||
vault_routes.py prefs_routes.py preset_routes.py
|
||||
signature_routes.py personal_routes.py hwfit_routes.py
|
||||
backup_routes.py cleanup_routes.py diagnostics_routes.py
|
||||
embedding_routes.py emoji_routes.py font_routes.py
|
||||
stt_routes.py tts_routes.py compare_routes.py
|
||||
editor_draft_routes.py chatgpt_subscription_routes.py admin_wipe_routes.py
|
||||
```
|
||||
|
||||
### `core/` (10 files)
|
||||
|
||||
```
|
||||
__init__.py constants.py database.py models.py
|
||||
auth.py middleware.py session_manager.py exceptions.py
|
||||
atomic_io.py platform_compat.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Key Import Relationships
|
||||
|
||||
```
|
||||
core/database.py ←── 102 importers (routes/*, src/*, core/*, tests/*)
|
||||
↑
|
||||
├── routes/auth_routes.py
|
||||
├── routes/email_routes.py
|
||||
├── src/builtin_actions.py
|
||||
├── src/task_scheduler.py
|
||||
├── src/tool_implementations.py (inline)
|
||||
└── ...97 more
|
||||
|
||||
src/tool_implementations.py ←── 17 importers
|
||||
↑
|
||||
├── src/agent_loop.py
|
||||
├── src/builtin_actions.py
|
||||
├── src/tool_index.py
|
||||
├── src/task_scheduler.py
|
||||
├── src/tool_policy.py
|
||||
└── ...12 more (mostly tests)
|
||||
|
||||
src/agent_loop.py ←── 22 importers
|
||||
↑
|
||||
├── src/tool_policy.py
|
||||
├── src/teacher_escalation.py
|
||||
├── src/bg_monitor.py
|
||||
├── src/task_scheduler.py
|
||||
└── 18 more (incl. tests)
|
||||
```
|
||||
# Architecture runtime inventory disposition
|
||||
|
||||
> [!NOTE]
|
||||
> The dated architecture runtime inventory has been classified and transferred
|
||||
> to [`discovery/architecture-runtime-inventory.md`](../discovery/architecture-runtime-inventory.md)
|
||||
> after verification of its stable facts.
|
||||
>
|
||||
> Stable runtime structure and subsystem boundaries were transferred to the
|
||||
> proposed canonical destination,
|
||||
> [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md), for maintainer review.
|
||||
|
||||
## Classification
|
||||
|
||||
| Content | Disposition |
|
||||
|---|---|
|
||||
| Stable runtime structure | Proposed canonical architecture document |
|
||||
| Stable subsystem boundaries | Proposed canonical architecture document |
|
||||
| Frontend module organization | `static/js/MODULE_SUMMARY.md` |
|
||||
| Counts, rankings, and generated metrics | Non-canonical discovery inventory |
|
||||
| Investigation context and unresolved questions | Non-canonical discovery material |
|
||||
| Refactor options and prioritization | Issues, Plane, or discovery material |
|
||||
|
||||
No dated metrics, prioritization, or investigation findings remain in this
|
||||
specification path.
|
||||
|
||||
## Review status
|
||||
|
||||
- Findings and scope have been validated.
|
||||
- Transfer verification has completed.
|
||||
- The final canonical status of `docs/ARCHITECTURE.md` requires maintainer review.
|
||||
- No implementation behaviour, test taxonomy, or runtime files were changed.
|
||||
|
||||
+7
-19
@@ -1237,27 +1237,15 @@ def _anthropic_rejects_temperature(model: str) -> bool:
|
||||
return False
|
||||
# `(?<![a-z])` anchors "opus" to a word boundary so a substring match like
|
||||
# `oct-opus`/`octopus-4-8` can't be read as Opus (it would otherwise strip
|
||||
# temperature). Both version components are capped at 1-2 digits and forbid a
|
||||
# trailing digit, so an 8-digit date can never be read as a version number:
|
||||
# `claude-opus-4-20250514` (Opus 4.0) parses as major-only rather than reading
|
||||
# `20250514` as a giant minor, and `claude-3-opus-20240229` (legacy Claude 3
|
||||
# Opus, date directly after "opus-") fails to match at all rather than reading
|
||||
# the date as a giant major. Dated 4.7+ snapshots (`claude-opus-4-7-20260201`)
|
||||
# keep their explicit minor and are still matched.
|
||||
#
|
||||
# The minor is optional and a missing minor reads as `.0`, so major-only ids
|
||||
# like `claude-opus-5` are correctly treated as >= 4.7 (issue #5753). Without
|
||||
# this, every Opus 5 call kept `temperature` and failed with HTTP 400 — visible
|
||||
# only on paths that pass a temperature, e.g. scheduled tasks inheriting
|
||||
# `stream_agent_loop`'s 0.3 default, which returned empty responses.
|
||||
match = re.search(
|
||||
r"(?<![a-z])opus[-_]?(\d{1,2})(?!\d)(?:[-_.](\d{1,2})(?!\d))?", model.lower()
|
||||
)
|
||||
# temperature). Cap the minor at 1-2 digits and forbid a trailing digit so a
|
||||
# dated id like `claude-opus-4-20250514` (Opus 4.0) parses as major-only (no
|
||||
# minor match, kept) instead of reading the date `20250514` as a giant minor
|
||||
# that would falsely test >= 4.7. Dated 4.7+ snapshots (`claude-opus-4-7-
|
||||
# 20260201`) keep their explicit minor and are still matched.
|
||||
match = re.search(r"(?<![a-z])opus[-_]?(\d+)[-_.](\d{1,2})(?!\d)", model.lower())
|
||||
if not match:
|
||||
return False
|
||||
major = int(match.group(1))
|
||||
minor = int(match.group(2)) if match.group(2) else 0
|
||||
return (major, minor) >= (4, 7)
|
||||
return (int(match.group(1)), int(match.group(2))) >= (4, 7)
|
||||
|
||||
# Reasoning effort level sent to Mistral thinking-capable models. Mistral's
|
||||
# API accepts "high", "medium", "low", "none" — see
|
||||
|
||||
+7
-13
@@ -758,36 +758,30 @@ export function mdToHtml(src, opts) {
|
||||
// Remove empty paragraphs
|
||||
s = s.replace(/<p><\/p>/g, '');
|
||||
|
||||
// Every restore below passes a function replacer rather than the block string
|
||||
// itself. With a string replacement, `String.replace` reads `$&`, `` $` ``,
|
||||
// `$'` and `$$` in the *replacement* as substitution patterns, so a restored
|
||||
// block containing them is corrupted: `$&` re-inserts the placeholder, `` $` ``
|
||||
// and `$'` splice in the surrounding document, and `$$` collapses to `$`. Those
|
||||
// sequences are ordinary content in fenced code (`perl -pe 's/x/$& y/'`,
|
||||
// `echo "$$USD"`). A function replacer inserts its return value verbatim.
|
||||
|
||||
// CRITICAL: Restore allowed HTML blocks first
|
||||
allowedHtmlBlocks.forEach((block, index) => {
|
||||
s = s.replace(`___ALLOWED_HTML_${index}___`, () => block);
|
||||
s = s.replace(`___ALLOWED_HTML_${index}___`, block);
|
||||
});
|
||||
|
||||
// Restore math blocks
|
||||
mathBlocks.forEach((block, index) => {
|
||||
s = s.replace(`___MATH_BLOCK_${index}___`, () => block);
|
||||
s = s.replace(`___MATH_BLOCK_${index}___`, block);
|
||||
});
|
||||
|
||||
// Restore mermaid diagram blocks
|
||||
mermaidBlocks.forEach((block, index) => {
|
||||
s = s.replace(`___MERMAID_BLOCK_${index}___`, () => block);
|
||||
s = s.replace(`___MERMAID_BLOCK_${index}___`, block);
|
||||
});
|
||||
|
||||
// CRITICAL: Restore code blocks at the end
|
||||
codeBlocks.forEach((block, index) => {
|
||||
s = s.replace(`___CODE_BLOCK_${index}___`, () => block);
|
||||
s = s.replace(`___CODE_BLOCK_${index}___`, block);
|
||||
});
|
||||
|
||||
// Restore inline code spans last, so placeholders carried inside restored
|
||||
// <a>/allowed-HTML blocks are resolved too.
|
||||
// <a>/allowed-HTML blocks are resolved too. The function replacer keeps the
|
||||
// escaped code literal — e.g. a shell snippet like `echo $1` is not treated
|
||||
// as a regex back-reference.
|
||||
inlineCodeBlocks.forEach((block, index) => {
|
||||
s = s.replace(`___INLINE_CODE_${index}___`, () => block);
|
||||
});
|
||||
|
||||
+22
-25
@@ -3031,14 +3031,12 @@ async function initEmailAccountsSettings() {
|
||||
const body = {
|
||||
name: el('eaf-name').value.trim() || el('eaf-from').value.trim(),
|
||||
from_address: el('eaf-from').value.trim(),
|
||||
display_name: el('eaf-display-name').value.trim(),
|
||||
imap_host: el('eaf-imap-host').value.trim(),
|
||||
imap_port: parseInt(el('eaf-imap-port').value) || 993,
|
||||
imap_user: el('eaf-imap-user').value.trim(),
|
||||
imap_starttls: el('eaf-imap-starttls').checked,
|
||||
smtp_host: el('eaf-smtp-host').value.trim(),
|
||||
smtp_port: parseInt(el('eaf-smtp-port').value) || 587,
|
||||
smtp_security: el('eaf-smtp-security').value,
|
||||
smtp_user: el('eaf-imap-user').value.trim(),
|
||||
};
|
||||
if (!body.name) { el('eaf-msg').textContent = 'Enter a Name or Email first'; el('eaf-msg').style.color = 'var(--red)'; return; }
|
||||
@@ -5790,30 +5788,29 @@ export function close() {
|
||||
window.history.replaceState(null, '', clean);
|
||||
const success = sp.has('email_oauth_success');
|
||||
const errMsg = sp.get('email_oauth_error') || '';
|
||||
// Open settings → integrations once the document is ready. This module owns
|
||||
// the open() API, so it does not need to wait for a window-level alias.
|
||||
function _showResult() {
|
||||
open('integrations');
|
||||
// Brief toast-style banner.
|
||||
const banner = document.createElement('div');
|
||||
banner.textContent = success
|
||||
? 'Google account connected — email is ready'
|
||||
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
|
||||
Object.assign(banner.style, {
|
||||
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
|
||||
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
|
||||
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
|
||||
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
|
||||
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
|
||||
});
|
||||
document.body.appendChild(banner);
|
||||
setTimeout(() => banner.remove(), 4000);
|
||||
}
|
||||
if (document.readyState === 'loading') {
|
||||
document.addEventListener('DOMContentLoaded', _showResult, { once: true });
|
||||
} else {
|
||||
_showResult();
|
||||
// Open settings → integrations after the app has initialised.
|
||||
function _tryOpen() {
|
||||
if (window.settingsModule && typeof window.settingsModule.open === 'function') {
|
||||
window.settingsModule.open('integrations');
|
||||
// Brief toast-style banner.
|
||||
const banner = document.createElement('div');
|
||||
banner.textContent = success
|
||||
? '✓ Google account connected — email is ready'
|
||||
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
|
||||
Object.assign(banner.style, {
|
||||
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
|
||||
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
|
||||
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
|
||||
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
|
||||
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
|
||||
});
|
||||
document.body.appendChild(banner);
|
||||
setTimeout(() => banner.remove(), 4000);
|
||||
} else {
|
||||
setTimeout(_tryOpen, 100);
|
||||
}
|
||||
}
|
||||
_tryOpen();
|
||||
})();
|
||||
|
||||
const settingsModule = { open, close, initIntegrations, initUnifiedIntegrations, syncAdminVisibility, refreshAiModelEndpoints };
|
||||
|
||||
+142
-175
@@ -1,202 +1,169 @@
|
||||
# Test Layout Inventory
|
||||
# Test layout inventory
|
||||
|
||||
> [!NOTE]
|
||||
> This document is a completed implementation snapshot for the first low-risk
|
||||
> test-directory reorganization. It does not propose additional file moves and
|
||||
> is not the canonical definition of the current test taxonomy.
|
||||
|
||||
- **Original inventory:** #3712
|
||||
- **Parent tracker:** #2523
|
||||
- **Implemented by:** #3842
|
||||
- **Status:** completed historical snapshot
|
||||
|
||||
- **Durable testing policy:** [`TESTING_STANDARD.md`](./TESTING_STANDARD.md)
|
||||
- **Current testing mechanics:** [`README.md`](./README.md)
|
||||
|
||||
## Disposition
|
||||
|
||||
This document is retained as a non-canonical historical record of the completed
|
||||
CLI test relocation.
|
||||
|
||||
Reusable testing rules are maintained in `TESTING_STANDARD.md`; current helper,
|
||||
taxonomy-runner, and execution mechanics are maintained in `README.md`, the test
|
||||
suite, and its supporting code.
|
||||
|
||||
Counts, residual-file observations, and path listings in this document are
|
||||
snapshot evidence only. They must not be maintained as durable repository truth.
|
||||
|
||||
Any additional test relocation requires a separately approved issue or work item.
|
||||
|
||||
## Purpose
|
||||
|
||||
Inventory for the first low-risk split of the flat `tests/` directory
|
||||
(issue #3712, parent #2523). This document only records *what* should move
|
||||
first and *why*; it moves nothing. The actual move is a separate, mechanical
|
||||
PR that relocates the listed files verbatim and changes no test content.
|
||||
This document preserves the reasoning, scope, and validation contract for the
|
||||
first mechanical reorganization of the formerly flat `tests/` directory.
|
||||
|
||||
The target layout and category definitions come from
|
||||
[`TESTING_STANDARD.md`](./TESTING_STANDARD.md); the collection-time markers
|
||||
come from [`_taxonomy.py`](./_taxonomy.py), which classifies by **filename
|
||||
tokens only** (paths are ignored, except the `tests/helpers/` rule). A file
|
||||
keeps its `area_*`/`sub_*` markers when moved into a subdirectory, and
|
||||
`conftest.py` discovers marker names recursively (`rglob`), so a move does not
|
||||
disturb marker registration or focused selection.
|
||||
The original change moved 28 tests classified as `area_cli` into `tests/cli/`
|
||||
without changing their assertions, helpers, taxonomy, or runtime behaviour.
|
||||
|
||||
## Current low-risk candidate groups
|
||||
Current testing guidance and classification rules are maintained in:
|
||||
|
||||
Groups whose tests need no route/app setup and no real DB/session setup:
|
||||
- [`README.md`](./README.md)
|
||||
- [`TESTING_STANDARD.md`](./TESTING_STANDARD.md)
|
||||
- [`_taxonomy.py`](./_taxonomy.py)
|
||||
|
||||
1. **CLI / script tests** (`area_cli`, 28 files) - load `scripts/` entry
|
||||
points via `tests.helpers.cli_loader.load_script`; DB access is stubbed
|
||||
with `tests.helpers.db_stubs` (`SessionLocal` is a plain stub attribute).
|
||||
No `TestClient`, no FastAPI app import, no SQLite files.
|
||||
2. **Helper self-tests** (`area_helpers`) - e.g. `test_helpers_import_state.py`,
|
||||
`test_db_stubs_helper.py`. Safe but tiny (two files), and they test the
|
||||
shared helpers from the #3685 audit (merged) that the rest of the suite
|
||||
depends on; little payoff as a first slice.
|
||||
3. **Pure unit / parsing tests** (`area_unit`) - `*_nonstring.py`,
|
||||
`*_nondict.py`, parsing tests. Large and heterogeneous; some touch
|
||||
provider/session modules, so the boundary is less crisp.
|
||||
4. **Static checks** - e.g. `test_readme_ascii_fenced.py`,
|
||||
`test_docs_no_orphan_images.py`. Safe but tiny and `uncategorized` in the
|
||||
taxonomy, so a move buys little and matches no existing marker.
|
||||
The current repository and test suite remain authoritative when this snapshot
|
||||
and the implementation differ.
|
||||
|
||||
Not candidates for the first move (per #3712 guidance): security/owner-scope
|
||||
tests, route/API tests, DB/session-heavy tests, auth/session concurrency
|
||||
tests, and the taxonomy/runner infrastructure tests that changed recently
|
||||
(#3491, #3556, #3659, #3711).
|
||||
## Why CLI tests were selected
|
||||
|
||||
## Recommended first move
|
||||
The CLI and script tests were selected as the first low-risk group because they
|
||||
formed a narrow, mechanically identifiable boundary:
|
||||
|
||||
**CLI / script tests → `tests/cli/`**
|
||||
- script entry points were loaded through shared CLI helpers;
|
||||
- database access used controlled stubs rather than persistent sessions;
|
||||
- the tests did not require FastAPI application or route setup;
|
||||
- the tests did not require a real database;
|
||||
- taxonomy classification could be derived consistently from filenames;
|
||||
- absolute helper imports remained valid after relocation.
|
||||
|
||||
Why this group over the alternatives:
|
||||
This reduced the risk relative to route, security, session, provider, and
|
||||
heterogeneous unit-test groups.
|
||||
|
||||
- Lowest coupling: every file imports only the script under test (via
|
||||
`cli_loader`) plus `tests.helpers` stubs - no app, no routes, no real DB.
|
||||
- Crisp, machine-checkable boundary: the set is exactly the files classified
|
||||
`area_cli` by `_taxonomy.py`, so before/after selection counts can be
|
||||
compared mechanically.
|
||||
- Already the planned target dir for this category in `TESTING_STANDARD.md`
|
||||
(`tests/cli/`).
|
||||
- Absolute imports (`from tests.helpers...`) and unique basenames mean no
|
||||
import-order or module-name collisions after the move.
|
||||
- Lower risk than helper self-tests (tiny group, little payoff), unit tests
|
||||
(fuzzy boundary), or anything security/route/session-shaped.
|
||||
## Completed implementation
|
||||
|
||||
## Files included in the first move
|
||||
Repository history records the implementation as:
|
||||
|
||||
The 28 files classified `area_cli` (verified against `_taxonomy.py`):
|
||||
- `83af3ca test: move area_cli tests into cli directory (#3842)`
|
||||
|
||||
Note: this inventory was refreshed against current `dev` after `tests/test_research_cli_status.py` was added to the `area_cli` set.
|
||||
The following 28 files were included in that completed move:
|
||||
|
||||
- `tests/test_calendar_cli_name.py`
|
||||
- `tests/test_contacts_cli_rows.py`
|
||||
- `tests/test_cookbook_cli_state.py`
|
||||
- `tests/test_docs_cli_content_length.py`
|
||||
- `tests/test_gallery_cli_album_count.py`
|
||||
- `tests/test_gallery_cli_preview.py`
|
||||
- `tests/test_logs_cli_resolve_nonstring.py`
|
||||
- `tests/test_mail_cli_read_empty_fetch.py`
|
||||
- `tests/test_mail_cli_recipients.py`
|
||||
- `tests/test_mcp_cli_env_serialize.py`
|
||||
- `tests/test_mcp_cli_json.py`
|
||||
- `tests/test_memory_cli_rows.py`
|
||||
- `tests/test_notes_cli_items.py`
|
||||
- `tests/test_personal_cli_rows.py`
|
||||
- `tests/test_preset_cli_invalid_entries.py`
|
||||
- `tests/test_preset_cli_set_corrupt_entry.py`
|
||||
- `tests/test_preset_cli_store.py`
|
||||
- `tests/test_research_cli_preview.py`
|
||||
- `tests/test_research_cli_status_filter.py`
|
||||
- `tests/test_research_cli_status.py`
|
||||
- `tests/test_research_cli_store.py`
|
||||
- `tests/test_sessions_cli.py`
|
||||
- `tests/test_signature_cli_export.py`
|
||||
- `tests/test_skills_cli_preview.py`
|
||||
- `tests/test_skills_cli_rows.py`
|
||||
- `tests/test_tasks_cli_preview.py`
|
||||
- `tests/test_theme_cli_store.py`
|
||||
- `tests/test_webhook_cli_mask.py`
|
||||
- `tests/cli/test_calendar_cli_name.py`
|
||||
- `tests/cli/test_contacts_cli_rows.py`
|
||||
- `tests/cli/test_cookbook_cli_state.py`
|
||||
- `tests/cli/test_docs_cli_content_length.py`
|
||||
- `tests/cli/test_gallery_cli_album_count.py`
|
||||
- `tests/cli/test_gallery_cli_preview.py`
|
||||
- `tests/cli/test_logs_cli_resolve_nonstring.py`
|
||||
- `tests/cli/test_mail_cli_read_empty_fetch.py`
|
||||
- `tests/cli/test_mail_cli_recipients.py`
|
||||
- `tests/cli/test_mcp_cli_env_serialize.py`
|
||||
- `tests/cli/test_mcp_cli_json.py`
|
||||
- `tests/cli/test_memory_cli_rows.py`
|
||||
- `tests/cli/test_notes_cli_items.py`
|
||||
- `tests/cli/test_personal_cli_rows.py`
|
||||
- `tests/cli/test_preset_cli_invalid_entries.py`
|
||||
- `tests/cli/test_preset_cli_set_corrupt_entry.py`
|
||||
- `tests/cli/test_preset_cli_store.py`
|
||||
- `tests/cli/test_research_cli_preview.py`
|
||||
- `tests/cli/test_research_cli_status.py`
|
||||
- `tests/cli/test_research_cli_status_filter.py`
|
||||
- `tests/cli/test_research_cli_store.py`
|
||||
- `tests/cli/test_sessions_cli.py`
|
||||
- `tests/cli/test_signature_cli_export.py`
|
||||
- `tests/cli/test_skills_cli_preview.py`
|
||||
- `tests/cli/test_skills_cli_rows.py`
|
||||
- `tests/cli/test_tasks_cli_preview.py`
|
||||
- `tests/cli/test_theme_cli_store.py`
|
||||
- `tests/cli/test_webhook_cli_mask.py`
|
||||
|
||||
## Files intentionally excluded
|
||||
## Original exclusions
|
||||
|
||||
- `tests/test_backup_cli_security.py` - classifies as `area_security`
|
||||
(security outranks cli in the taxonomy); moving it into `tests/cli/` would
|
||||
make the directory disagree with its marker. It belongs with the security
|
||||
group in a later phase.
|
||||
- `tests/test_run_focus.py`, `tests/test_taxonomy.py` - taxonomy/runner
|
||||
infrastructure tests, recently changed (#3556, #3659); they also pin
|
||||
flat-layout paths (e.g. `tests/test_auth_config_lock_concurrency.py` in
|
||||
`test_run_focus.py`), so they stay put.
|
||||
- Script-like but `uncategorized` files - `test_pr_blocker_audit.py`,
|
||||
`test_update_database_script.py`, `test_windows_update_script.py`,
|
||||
`test_setup_admin_user.py`, `test_amd_gpu_check_args.py`, `test_hwfit_*.py`.
|
||||
They exercise `scripts/` too, but moving them would make `tests/cli/`
|
||||
diverge from the `area_cli` marker set. Reclassify or move them in a later,
|
||||
separate slice.
|
||||
- Everything else (security, routes, services, unit, js, helpers) - out of
|
||||
scope for the first move by design.
|
||||
The original inventory deliberately excluded:
|
||||
|
||||
## How this was verified
|
||||
- `tests/test_backup_cli_security.py`, because its security classification
|
||||
outranked the CLI token in its filename;
|
||||
- taxonomy and focused-runner infrastructure, including
|
||||
`tests/test_taxonomy.py` and `tests/test_run_focus.py`;
|
||||
- script-oriented tests not classified as `area_cli`;
|
||||
- route, service, security, session, JavaScript, helper, and broad unit-test
|
||||
groups.
|
||||
|
||||
Read-only checks, run from the repo root on this branch. Note the real API is
|
||||
`classify_test_path` (there is no `classify_test_file`).
|
||||
These exclusions kept the implementation mechanical and prevented it from
|
||||
mixing taxonomy changes, assertion changes, helper extraction, or unrelated
|
||||
test-suite restructuring.
|
||||
|
||||
## Current repository observation
|
||||
|
||||
The current taxonomy classifies 30 files as `area_cli`:
|
||||
|
||||
| Location | Files |
|
||||
|---|---:|
|
||||
| `tests/cli/` | 28 |
|
||||
| Flat `tests/` directory | 2 |
|
||||
|
||||
The two currently classified CLI tests outside `tests/cli/` are:
|
||||
|
||||
- `tests/test_calendar_cli_overlap.py`
|
||||
- `tests/test_memory_cli_add_nondict.py`
|
||||
|
||||
These files were not part of the original 28-file implementation.
|
||||
|
||||
Their presence is recorded solely to keep this snapshot accurate. This
|
||||
documentation update does not recommend moving them, changing their taxonomy,
|
||||
or opening implementation work. Any future test-layout change requires separate
|
||||
maintainer agreement, scope, and validation.
|
||||
|
||||
## Validation contract preserved by the original move
|
||||
|
||||
The completed implementation was designed to preserve:
|
||||
|
||||
- identical `area_cli` selection before and after relocation;
|
||||
- successful direct execution of the relocated tests;
|
||||
- successful whole-suite collection;
|
||||
- unchanged taxonomy and focused-runner behaviour;
|
||||
- no stale references to the former flat paths;
|
||||
- no production-code changes.
|
||||
|
||||
Relevant current verification commands include:
|
||||
|
||||
```bash
|
||||
# Compute the area_cli set and confirm test_backup_cli_security.py is
|
||||
# area_security. Expected: 28 files, then "security".
|
||||
./venv/bin/python - <<'PY'
|
||||
from pathlib import Path
|
||||
from tests._taxonomy import classify_test_path
|
||||
|
||||
cli = [p for p in sorted(Path("tests").glob("test_*.py"))
|
||||
if classify_test_path(p).area == "cli"]
|
||||
print(len(cli))
|
||||
for p in cli:
|
||||
print(p)
|
||||
print(classify_test_path("tests/test_backup_cli_security.py").area)
|
||||
PY
|
||||
|
||||
# Coupling check across the CLI files. Expected: the only hits are
|
||||
# "SessionLocal" as stub attribute names passed to tests.helpers.db_stubs;
|
||||
# no TestClient, FastAPI, create_app, sqlite, or dependency_overrides.
|
||||
rg -n "TestClient|FastAPI|create_app|SessionLocal|sqlite|dependency_overrides" \
|
||||
tests/test_*cli*.py tests/test_sessions_cli.py
|
||||
|
||||
# Hard-coded flat paths to the exact CLI files outside tests/. Expected: no matches.
|
||||
./venv/bin/python - <<'PY2' > /tmp/area_cli_paths.txt
|
||||
from pathlib import Path
|
||||
from tests._taxonomy import classify_test_path
|
||||
|
||||
for path in sorted(Path("tests").glob("test_*.py")):
|
||||
if classify_test_path(path).area == "cli":
|
||||
print(path)
|
||||
PY2
|
||||
|
||||
rg -n -F -f /tmp/area_cli_paths.txt .github scripts docs \
|
||||
tests/README.md tests/TESTING_STANDARD.md pyproject.toml 2>/dev/null || true
|
||||
venv/bin/python tests/run_focus.py --dry-run --area cli
|
||||
venv/bin/python -m pytest -m area_cli -q
|
||||
venv/bin/python -m pytest tests/cli/ -q
|
||||
venv/bin/python -m pytest tests/test_taxonomy.py tests/test_run_focus.py -q
|
||||
venv/bin/python -m pytest --collect-only -q
|
||||
```
|
||||
|
||||
Also checked by reading the code: `tests/conftest.py` registers sub-markers
|
||||
from a recursive `rglob` scan, and `tests/_taxonomy.py` classifies by filename
|
||||
tokens only (plus the `tests/helpers/` directory rule), so the markers of the
|
||||
28 files do not change when they move into `tests/cli/`.
|
||||
|
||||
## Validation for the future move PR
|
||||
|
||||
Run with the project venv (`./venv/bin/python`); system `python3` may miss
|
||||
pinned deps. Before the move, record the baseline; after, compare:
|
||||
|
||||
```bash
|
||||
# Selection must match the 28 files before and after the move.
|
||||
./venv/bin/python tests/run_focus.py --dry-run --area cli
|
||||
./venv/bin/python -m pytest -m area_cli -q
|
||||
|
||||
# Moved files pass when targeted directly.
|
||||
./venv/bin/python -m pytest tests/cli/ -q
|
||||
|
||||
# Whole-suite collection still succeeds (catches import/path breakage).
|
||||
./venv/bin/python -m pytest --collect-only -q
|
||||
|
||||
# Taxonomy/runner infrastructure is unaffected.
|
||||
./venv/bin/python -m pytest tests/test_taxonomy.py tests/test_run_focus.py -q
|
||||
|
||||
# No stale flat-path references to the moved files. Expected: no matches
|
||||
# outside tests/cli/ itself.
|
||||
./venv/bin/python - <<'PY2' > /tmp/area_cli_paths.txt
|
||||
from pathlib import Path
|
||||
from tests._taxonomy import classify_test_path
|
||||
|
||||
for path in sorted(Path("tests").glob("test_*.py")):
|
||||
if classify_test_path(path).area == "cli":
|
||||
print(path)
|
||||
PY2
|
||||
|
||||
rg -n -F -f /tmp/area_cli_paths.txt .github scripts docs \
|
||||
tests/README.md tests/TESTING_STANDARD.md pyproject.toml 2>/dev/null || true
|
||||
```
|
||||
|
||||
Pass criteria: identical test counts for `-m area_cli` before/after, zero
|
||||
collection errors, and no changes outside the moved files.
|
||||
These commands describe validation coverage. This snapshot does not claim that
|
||||
they were executed as part of the present documentation update.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No file moves, renames, or deletions in this PR.
|
||||
- No changes to `conftest.py`, `_taxonomy.py`, `run_focus.py`, helpers,
|
||||
markers, CI workflows, or production code.
|
||||
- No recommendation to split the whole suite at once; later groups get their
|
||||
own inventory-then-move slices.
|
||||
This document does not:
|
||||
|
||||
- propose moving or renaming tests;
|
||||
- redefine taxonomy classifications;
|
||||
- change test behaviour or assertions;
|
||||
- prescribe the next test-suite refactor;
|
||||
- replace the testing standard;
|
||||
- create implementation work from the two current residual files.
|
||||
|
||||
+181
-148
@@ -1,5 +1,13 @@
|
||||
# Oversized Test File Split Plan
|
||||
|
||||
> [!NOTE]
|
||||
> This is generated planning material based on the current test tree.
|
||||
>
|
||||
> It is non-canonical and disposable. Durable testing policy belongs in
|
||||
> `tests/TESTING_STANDARD.md`; active split selection and prioritization
|
||||
> belong in approved issues or the project tracker.
|
||||
> It does not define runtime behaviour or approve any test-file split.
|
||||
|
||||
## Purpose
|
||||
|
||||
This document plans future oversized test-file splits using current repo data.
|
||||
@@ -26,8 +34,8 @@ Future split PRs must still inspect each file manually before editing.
|
||||
|
||||
## Current summary
|
||||
|
||||
- test files scanned: 583
|
||||
- collected pytest items counted: 3586
|
||||
- test files scanned: 746
|
||||
- collected pytest items counted: 4764
|
||||
- large-file threshold: 300 lines
|
||||
- large-collected threshold: 20 collected items
|
||||
|
||||
@@ -35,123 +43,123 @@ Area distribution:
|
||||
|
||||
| Value | Files |
|
||||
|---|---:|
|
||||
| cli | 28 |
|
||||
| cli | 30 |
|
||||
| helpers | 1 |
|
||||
| js | 39 |
|
||||
| routes | 23 |
|
||||
| security | 77 |
|
||||
| services | 144 |
|
||||
| uncategorized | 234 |
|
||||
| unit | 37 |
|
||||
| js | 49 |
|
||||
| routes | 35 |
|
||||
| security | 88 |
|
||||
| services | 197 |
|
||||
| uncategorized | 306 |
|
||||
| unit | 40 |
|
||||
|
||||
Sub-area distribution:
|
||||
|
||||
| Value | Files |
|
||||
|---|---:|
|
||||
| api | 6 |
|
||||
| api | 8 |
|
||||
| atomic | 3 |
|
||||
| auth | 9 |
|
||||
| calendar | 10 |
|
||||
| cli | 28 |
|
||||
| confinement | 7 |
|
||||
| cookbook | 13 |
|
||||
| document | 11 |
|
||||
| email | 12 |
|
||||
| embedding | 3 |
|
||||
| gallery | 5 |
|
||||
| auth | 11 |
|
||||
| calendar | 12 |
|
||||
| cli | 30 |
|
||||
| confinement | 8 |
|
||||
| cookbook | 20 |
|
||||
| document | 13 |
|
||||
| email | 23 |
|
||||
| embedding | 6 |
|
||||
| gallery | 6 |
|
||||
| history | 3 |
|
||||
| js | 39 |
|
||||
| llm | 16 |
|
||||
| mcp | 8 |
|
||||
| memory | 15 |
|
||||
| js | 49 |
|
||||
| llm | 21 |
|
||||
| mcp | 12 |
|
||||
| memory | 17 |
|
||||
| nondict | 7 |
|
||||
| nonstring | 22 |
|
||||
| owner | 14 |
|
||||
| owner_scope | 23 |
|
||||
| parse | 4 |
|
||||
| provider | 6 |
|
||||
| research | 16 |
|
||||
| nonstring | 23 |
|
||||
| owner | 15 |
|
||||
| owner_scope | 27 |
|
||||
| parse | 5 |
|
||||
| provider | 16 |
|
||||
| research | 17 |
|
||||
| route | 6 |
|
||||
| routes | 9 |
|
||||
| scheduler | 3 |
|
||||
| routes | 19 |
|
||||
| scheduler | 4 |
|
||||
| scope | 5 |
|
||||
| security | 9 |
|
||||
| session | 16 |
|
||||
| ssrf | 3 |
|
||||
| webhook | 3 |
|
||||
| session | 18 |
|
||||
| ssrf | 6 |
|
||||
| webhook | 4 |
|
||||
| xss | 5 |
|
||||
|
||||
Values below 2 files: 244 values covering 244 files.
|
||||
Values below 2 files: 318 values covering 318 files.
|
||||
|
||||
## Top files by collected pytest items
|
||||
|
||||
| File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
|
||||
|---|---:|---:|---:|---:|---|---|---|
|
||||
| `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
|
||||
| `tests/test_provider_classification.py` | 188 | 67 | 21 | 4 | services | provider | - |
|
||||
| `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
|
||||
| `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
|
||||
| `tests/test_model_routes.py` | 2099 | 163 | 134 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_security_regressions.py` | 1515 | 98 | 74 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
|
||||
| `tests/test_cookbook_helpers.py` | 958 | 69 | 69 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
|
||||
| `tests/test_shell_routes.py` | 538 | 69 | 53 | 9 | routes | routes | route/api, import-state, filesystem |
|
||||
| `tests/test_endpoint_probing.py` | 497 | 61 | 36 | 7 | uncategorized | endpoint_probing | route/api, db/session, import-state |
|
||||
| `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
|
||||
| `tests/test_provider_endpoints.py` | 241 | 58 | 18 | 1 | services | provider | subprocess/script |
|
||||
| `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
|
||||
| `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
|
||||
| `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
|
||||
| `tests/test_llm_core_temperature.py` | 196 | 41 | 17 | 0 | services | llm | - |
|
||||
| `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
|
||||
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | 26 | 0 | security | confinement | route/api, security, filesystem, async/threading |
|
||||
| `tests/test_run_focus.py` | 492 | 55 | 52 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
|
||||
| `tests/test_agent_loop.py` | 500 | 54 | 54 | 5 | uncategorized | agent_loop | db/session, import-state |
|
||||
| `tests/test_manage_mcp_command_allowlist.py` | 168 | 45 | 10 | 0 | services | mcp | db/session, subprocess/script, async/threading, ui/static |
|
||||
| `tests/test_provider_classification.py` | 115 | 42 | 9 | 2 | services | provider | - |
|
||||
| `tests/test_review_regressions.py` | 1429 | 41 | 41 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_chat_helpers.py` | 582 | 37 | 24 | 0 | uncategorized | chat_helpers | route/api, filesystem, async/threading |
|
||||
| `tests/test_model_context.py` | 314 | 36 | 34 | 4 | uncategorized | model_context | db/session, import-state |
|
||||
| `tests/test_email_oauth.py` | 798 | 34 | 29 | 0 | services | email | route/api, db/session, security, async/threading |
|
||||
| `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | 6 | 0 | services | llm | db/session |
|
||||
| `tests/test_chat_helpers.py` | 264 | 31 | 18 | 0 | uncategorized | chat_helpers | route/api |
|
||||
| `tests/test_provider_detection.py` | 148 | 31 | 31 | 5 | services | provider | - |
|
||||
| `tests/test_model_context.py` | 251 | 30 | 30 | 4 | uncategorized | model_context | db/session, import-state |
|
||||
| `tests/test_endpoint_resolver.py` | 148 | 30 | 30 | 6 | uncategorized | endpoint_resolver | - |
|
||||
| `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
|
||||
| `tests/test_redos_verdict_continuation.py` | 80 | 30 | 5 | 0 | uncategorized | redos_verdict_continuation | route/api |
|
||||
| `tests/test_upload_limits_centralized.py` | 110 | 29 | 5 | 0 | uncategorized | upload_limits_centralized | import-state, filesystem |
|
||||
| `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
|
||||
| `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_cookbook_docker_access.py` | 339 | 28 | 13 | 0 | services | cookbook | route/api, filesystem, async/threading |
|
||||
| `tests/test_rename_user_owner_sync.py` | 740 | 27 | 27 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_taxonomy.py` | 151 | 27 | 17 | 0 | uncategorized | taxonomy | security, ui/static |
|
||||
| `tests/test_endpoint_resolver_urls.py` | 95 | 27 | 23 | 3 | uncategorized | endpoint_resolver_urls | - |
|
||||
| `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
|
||||
| `tests/test_taxonomy.py` | 145 | 26 | 16 | 0 | uncategorized | taxonomy | security, ui/static |
|
||||
| `tests/test_tool_path_confinement.py` | 282 | 24 | 24 | 0 | security | confinement | import-state, filesystem, async/threading |
|
||||
| `tests/test_copilot.py` | 170 | 23 | 16 | 0 | uncategorized | copilot | - |
|
||||
| `tests/test_research_utils.py` | 97 | 23 | 23 | 2 | services | research | - |
|
||||
| `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_tool_support_heuristic.py` | 166 | 22 | 22 | 3 | uncategorized | tool_support_heuristic | - |
|
||||
| `tests/test_platform_compat.py` | 318 | 21 | 21 | 0 | uncategorized | platform_compat | import-state, filesystem, subprocess/script |
|
||||
| `tests/test_workspace_confine.py` | 527 | 25 | 25 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
|
||||
| `tests/test_chat_route_tool_policy.py` | 344 | 25 | 23 | 0 | routes | route | filesystem, ui/static |
|
||||
| `tests/test_tool_path_confinement.py` | 303 | 25 | 25 | 0 | security | confinement | import-state, filesystem, async/threading |
|
||||
| `tests/test_copilot.py` | 182 | 25 | 18 | 0 | uncategorized | copilot | - |
|
||||
| `tests/test_llm_core_temperature_reasoning.py` | 133 | 25 | 9 | 0 | services | llm | - |
|
||||
| `tests/test_provider_endpoints_url_building.py` | 86 | 25 | 3 | 0 | services | provider | - |
|
||||
| `tests/test_bg_job_tools.py` | 174 | 23 | 16 | 0 | uncategorized | bg_job_tools | filesystem, async/threading |
|
||||
|
||||
## Top files by physical line count
|
||||
|
||||
| File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
|
||||
|---|---:|---:|---:|---:|---|---|---|
|
||||
| `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
|
||||
| `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
|
||||
| `tests/test_model_routes.py` | 2099 | 163 | 134 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_security_regressions.py` | 1515 | 98 | 74 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
|
||||
| `tests/test_review_regressions.py` | 1429 | 41 | 41 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
|
||||
| `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
|
||||
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
|
||||
| `tests/test_cookbook_helpers.py` | 958 | 69 | 69 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
|
||||
| `tests/test_upload_handler_cleanup.py` | 831 | 13 | 13 | 0 | uncategorized | upload_handler_cleanup | route/api, db/session, filesystem, async/threading |
|
||||
| `tests/test_email_oauth.py` | 798 | 34 | 29 | 0 | services | email | route/api, db/session, security, async/threading |
|
||||
| `tests/test_rename_user_owner_sync.py` | 740 | 27 | 27 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_email_owner_scope.py` | 656 | 13 | 13 | 0 | security | owner_scope | route/api, db/session, filesystem, async/threading |
|
||||
| `tests/test_model_capability_readers.py` | 646 | 20 | 20 | 0 | uncategorized | model_capability_readers | - |
|
||||
| `tests/test_chat_helpers.py` | 582 | 37 | 24 | 0 | uncategorized | chat_helpers | route/api, filesystem, async/threading |
|
||||
| `tests/test_api_token_routes.py` | 578 | 17 | 17 | 0 | routes | api_routes | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
|
||||
| `tests/test_email_owner_scope.py` | 474 | 9 | 9 | 0 | security | owner_scope | route/api, db/session, filesystem, async/threading |
|
||||
| `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
|
||||
| `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
|
||||
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | 26 | 0 | security | confinement | route/api, security, filesystem, async/threading |
|
||||
| `tests/test_shell_routes.py` | 538 | 69 | 53 | 9 | routes | routes | route/api, import-state, filesystem |
|
||||
| `tests/test_workspace_confine.py` | 527 | 25 | 25 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
|
||||
| `tests/test_agent_loop.py` | 500 | 54 | 54 | 5 | uncategorized | agent_loop | db/session, import-state |
|
||||
| `tests/test_endpoint_probing.py` | 497 | 61 | 36 | 7 | uncategorized | endpoint_probing | route/api, db/session, import-state |
|
||||
| `tests/test_run_focus.py` | 492 | 55 | 52 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
|
||||
| `tests/test_tool_policy.py` | 475 | 18 | 18 | 0 | uncategorized | tool_policy | import-state, async/threading |
|
||||
| `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | 8 | 0 | uncategorized | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_email_test_connection_oauth.py` | 436 | 9 | 7 | 0 | services | email | route/api, db/session, async/threading |
|
||||
| `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
|
||||
| `tests/test_endpoint_owner_scope_followup.py` | 414 | 11 | 11 | 0 | security | owner_scope | route/api, db/session, filesystem |
|
||||
| `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
|
||||
| `tests/test_api_chat_security.py` | 404 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_imap_leak_fixes.py` | 404 | 15 | 15 | 0 | uncategorized | imap_leak_fixes | route/api, db/session, security, filesystem |
|
||||
| `tests/test_companion_readonly.py` | 402 | 17 | 17 | 0 | uncategorized | companion_readonly | db/session, import-state |
|
||||
| `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
|
||||
| `tests/test_upload_handler_atomicity.py` | 401 | 9 | 9 | 0 | uncategorized | upload_handler_atomicity | filesystem, async/threading |
|
||||
| `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
|
||||
| `tests/test_auth_regressions.py` | 375 | 15 | 15 | 0 | security | auth | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_calendar_owner_scope.py` | 345 | 7 | 7 | 0 | security | owner_scope | route/api, db/session, import-state, filesystem, async/threading, ui/static |
|
||||
| `tests/test_null_owner_gates.py` | 342 | 20 | 20 | 0 | security | owner | route/api, db/session, import-state |
|
||||
| `tests/test_agent_migration_manifest.py` | 340 | 15 | 15 | 0 | uncategorized | agent_migration_manifest | import-state, filesystem |
|
||||
| `tests/test_calendar_recurrence.py` | 338 | 19 | 19 | 0 | services | calendar | - |
|
||||
| `tests/test_tool_policy.py` | 330 | 13 | 13 | 0 | uncategorized | tool_policy | import-state, async/threading |
|
||||
| `tests/test_workspace_confine.py` | 328 | 18 | 18 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
|
||||
| `tests/test_diffusion_server_security.py` | 325 | 14 | 14 | 0 | security | security | route/api, import-state, security, filesystem, async/threading, ui/static |
|
||||
| `tests/test_auth_regressions.py` | 370 | 15 | 15 | 0 | security | auth | route/api, db/session, import-state, async/threading |
|
||||
| `tests/test_calendar_recurrence.py` | 356 | 20 | 20 | 0 | services | calendar | - |
|
||||
| `tests/test_task_cookbook_admin_gate.py` | 350 | 10 | 10 | 0 | services | cookbook | route/api, db/session, import-state, filesystem, async/threading |
|
||||
|
||||
## Split planning candidates
|
||||
|
||||
@@ -165,53 +173,69 @@ These are planning candidates only. A later split PR still needs a focused manua
|
||||
|
||||
| File | Why included | Setup/risk signals | Suggested handling |
|
||||
|---|---|---|---|
|
||||
| `tests/test_model_routes.py` | 1778 lines, 139 collected tests | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_security_regressions.py` | 1224 lines, 92 collected tests | route/api, db/session, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_provider_classification.py` | 67 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_cookbook_helpers.py` | 912 lines, 65 collected tests | route/api, filesystem, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_shell_routes.py` | 481 lines, 63 collected tests | route/api, import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_model_routes.py` | 2099 lines, 163 collected tests | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_security_regressions.py` | 1515 lines, 98 collected tests | route/api, db/session, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_cookbook_helpers.py` | 958 lines, 69 collected tests | route/api, filesystem, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_shell_routes.py` | 538 lines, 69 collected tests | route/api, import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_endpoint_probing.py` | 497 lines, 61 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_pr_blocker_audit.py` | 964 lines, 58 collected tests | import-state, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_provider_endpoints.py` | 58 collected tests | subprocess/script | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_agent_loop.py` | 469 lines, 52 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_service_health.py` | 472 lines, 47 collected tests | async/threading | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_run_focus.py` | 399 lines, 47 collected tests | security, filesystem, subprocess/script, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_llm_core_temperature.py` | 41 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_endpoint_probing.py` | 411 lines, 34 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_research_routes_path_confinement.py` | 565 lines, 58 collected tests | route/api, security, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_run_focus.py` | 492 lines, 55 collected tests | security, filesystem, subprocess/script, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_agent_loop.py` | 500 lines, 54 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_manage_mcp_command_allowlist.py` | 45 collected tests | db/session, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_provider_classification.py` | 42 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_review_regressions.py` | 1429 lines, 41 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_chat_helpers.py` | 582 lines, 37 collected tests | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_model_context.py` | 314 lines, 36 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_email_oauth.py` | 798 lines, 34 collected tests | route/api, db/session, security, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_llm_core_anthropic_temp_omit.py` | 32 collected tests | db/session | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_chat_helpers.py` | 31 collected tests | route/api | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_provider_detection.py` | 31 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_model_context.py` | 30 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_endpoint_resolver.py` | 30 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_embedding_lanes.py` | 1104 lines, 29 collected tests | filesystem | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_redos_verdict_continuation.py` | 30 collected tests | route/api | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_upload_limits_centralized.py` | 29 collected tests | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_email_oauth.py` | 580 lines, 28 collected tests | route/api, db/session, security, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_review_regressions.py` | 930 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_rename_user_owner_sync.py` | 686 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_cookbook_docker_access.py` | 339 lines, 28 collected tests | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_rename_user_owner_sync.py` | 740 lines, 27 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_taxonomy.py` | 27 collected tests | security, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_endpoint_resolver_urls.py` | 27 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_helpers_import_state.py` | 426 lines, 26 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_taxonomy.py` | 26 collected tests | security, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_tool_path_confinement.py` | 24 collected tests | import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_copilot.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_workspace_confine.py` | 527 lines, 25 collected tests | route/api, filesystem, subprocess/script, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_chat_route_tool_policy.py` | 344 lines, 25 collected tests | filesystem, ui/static | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_tool_path_confinement.py` | 303 lines, 25 collected tests | import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_copilot.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_llm_core_temperature_reasoning.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_provider_endpoints_url_building.py` | 25 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_bg_job_tools.py` | 23 collected tests | filesystem, async/threading | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_research_utils.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_api_chat_security.py` | 401 lines, 22 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_api_chat_security.py` | 404 lines, 22 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_tool_support_heuristic.py` | 22 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_platform_compat.py` | 318 lines, 21 collected tests | import-state, filesystem, subprocess/script | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_platform_compat.py` | 322 lines, 21 collected tests | import-state, filesystem, subprocess/script | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_context_compactor.py` | 21 collected tests | db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_prompt_security.py` | 21 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_null_owner_gates.py` | 342 lines, 20 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_fenced_inline_args.py` | 21 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_cache_affinity_local_only.py` | 21 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_model_capability_readers.py` | 646 lines, 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_calendar_recurrence.py` | 356 lines, 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_null_owner_gates.py` | 332 lines, 20 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_gallery_endpoint_hardening.py` | 326 lines, 20 collected tests | route/api, security, filesystem, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_redos_llm_parsers.py` | 20 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
|
||||
| `tests/test_code_nav_tools.py` | 20 collected tests | db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_youtube_handler_consolidation.py` | 20 collected tests | route/api, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_calendar_recurrence.py` | 338 lines | No obvious setup signals from static scan. | Plan split boundaries before editing. |
|
||||
| `tests/test_workspace_confine.py` | 328 lines | route/api, filesystem, subprocess/script, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_tool_policy.py` | 475 lines | import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_fenced_example_not_executed_for_native_models.py` | 345 lines | async/threading | Plan split boundaries before editing. |
|
||||
| `tests/test_api_token_routes.py` | 578 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_companion_readonly.py` | 402 lines | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_set_admin.py` | 317 lines | route/api, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_llm_core_ollama.py` | 324 lines | async/threading | Plan split boundaries before editing. |
|
||||
| `tests/test_imap_leak_fixes.py` | 404 lines | route/api, db/session, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_auth_regressions.py` | 375 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_auth_regressions.py` | 370 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_agent_migration_manifest.py` | 340 lines | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_mcp_email_decode_header_spaces.py` | 327 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_diffusion_server_security.py` | 325 lines | route/api, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_tool_policy.py` | 330 lines | import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_upload_handler_cleanup.py` | 831 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_email_owner_scope.py` | 656 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_upload_routes_owner_scope.py` | 344 lines | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_endpoint_owner_scope_followup.py` | 414 lines | route/api, db/session, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_upload_routes_owner_scope.py` | 315 lines | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_email_owner_scope.py` | 474 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_composer_arrow_up_recall_js.py` | 308 lines | filesystem, subprocess/script, ui/static | Plan split boundaries before editing. |
|
||||
| `tests/test_task_cookbook_admin_gate.py` | 350 lines | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_email_test_connection_oauth.py` | 436 lines | route/api, db/session, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_upload_handler_atomicity.py` | 401 lines | filesystem, async/threading | Plan split boundaries before editing. |
|
||||
| `tests/test_kv_cache_invalidation_2927.py` | 463 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
| `tests/test_calendar_owner_scope.py` | 345 lines | route/api, db/session, import-state, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
|
||||
@@ -226,28 +250,35 @@ Before using any of these files as a split target, first decide whether the taxo
|
||||
|
||||
| File | Lines | Collected tests | Sub-area | Signals | Suggested follow-up |
|
||||
|---|---:|---:|---|---|---|
|
||||
| `tests/test_endpoint_probing.py` | 497 | 61 | endpoint_probing | route/api, db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_pr_blocker_audit.py` | 964 | 58 | pr_blocker_audit | import-state, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_agent_loop.py` | 469 | 52 | agent_loop | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_service_health.py` | 472 | 47 | service_health | async/threading | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_run_focus.py` | 399 | 47 | run_focus | security, filesystem, subprocess/script, ui/static | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_endpoint_probing.py` | 411 | 34 | endpoint_probing | route/api, db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_chat_helpers.py` | 264 | 31 | chat_helpers | route/api | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_model_context.py` | 251 | 30 | model_context | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_endpoint_resolver.py` | 148 | 30 | endpoint_resolver | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_run_focus.py` | 492 | 55 | run_focus | security, filesystem, subprocess/script, ui/static | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_agent_loop.py` | 500 | 54 | agent_loop | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_review_regressions.py` | 1429 | 41 | review_regressions | route/api, db/session, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_chat_helpers.py` | 582 | 37 | chat_helpers | route/api, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_model_context.py` | 314 | 36 | model_context | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_redos_verdict_continuation.py` | 80 | 30 | redos_verdict_continuation | route/api | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_upload_limits_centralized.py` | 110 | 29 | upload_limits_centralized | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_review_regressions.py` | 930 | 26 | review_regressions | route/api, db/session, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_taxonomy.py` | 145 | 26 | taxonomy | security, ui/static | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_copilot.py` | 170 | 23 | copilot | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_taxonomy.py` | 151 | 27 | taxonomy | security, ui/static | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_endpoint_resolver_urls.py` | 95 | 27 | endpoint_resolver_urls | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_workspace_confine.py` | 527 | 25 | workspace_confine | route/api, filesystem, subprocess/script, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_copilot.py` | 182 | 25 | copilot | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_bg_job_tools.py` | 174 | 23 | bg_job_tools | filesystem, async/threading | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_tool_support_heuristic.py` | 166 | 22 | tool_support_heuristic | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_platform_compat.py` | 318 | 21 | platform_compat | import-state, filesystem, subprocess/script | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_platform_compat.py` | 322 | 21 | platform_compat | import-state, filesystem, subprocess/script | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_context_compactor.py` | 233 | 21 | context_compactor | db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_fenced_inline_args.py` | 186 | 21 | fenced_inline_args | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_cache_affinity_local_only.py` | 104 | 21 | cache_affinity_local_only | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_model_capability_readers.py` | 646 | 20 | model_capability_readers | - | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_code_nav_tools.py` | 198 | 20 | code_nav_tools | db/session, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_youtube_handler_consolidation.py` | 104 | 20 | youtube_handler_consolidation | route/api, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_workspace_confine.py` | 328 | 18 | workspace_confine | route/api, filesystem, subprocess/script, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_tool_policy.py` | 475 | 18 | tool_policy | import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_fenced_example_not_executed_for_native_models.py` | 345 | 18 | fenced_example_not_executed_for_native_models | async/threading | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_companion_readonly.py` | 402 | 17 | companion_readonly | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_set_admin.py` | 317 | 17 | set_admin | route/api, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_imap_leak_fixes.py` | 404 | 15 | imap_leak_fixes | route/api, db/session, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_agent_migration_manifest.py` | 340 | 15 | agent_migration_manifest | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_tool_policy.py` | 330 | 13 | tool_policy | import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_upload_handler_cleanup.py` | 831 | 13 | upload_handler_cleanup | route/api, db/session, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
| `tests/test_upload_handler_atomicity.py` | 401 | 9 | upload_handler_atomicity | filesystem, async/threading | Review taxonomy mapping before using as a split target. |
|
||||
| `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
|
||||
|
||||
@@ -259,14 +290,16 @@ Files still in the `uncategorized` taxonomy area are listed separately below so
|
||||
|
||||
| File | Lines | Collected tests | Area | Sub-area | Signals | Why this is a candidate |
|
||||
|---|---:|---:|---|---|---|---|
|
||||
| `tests/test_provider_classification.py` | 188 | 67 | services | provider | - | 67 collected tests |
|
||||
| `tests/test_provider_endpoints.py` | 241 | 58 | services | provider | subprocess/script | 58 collected tests |
|
||||
| `tests/test_llm_core_temperature.py` | 196 | 41 | services | llm | - | 41 collected tests |
|
||||
| `tests/test_provider_detection.py` | 148 | 31 | services | provider | - | 31 collected tests |
|
||||
| `tests/test_embedding_lanes.py` | 1104 | 29 | services | embedding | filesystem | 1104 lines, 29 collected tests |
|
||||
| `tests/test_provider_classification.py` | 115 | 42 | services | provider | - | 42 collected tests |
|
||||
| `tests/test_chat_route_tool_policy.py` | 344 | 25 | routes | route | filesystem, ui/static | 344 lines, 25 collected tests |
|
||||
| `tests/test_llm_core_temperature_reasoning.py` | 133 | 25 | services | llm | - | 25 collected tests |
|
||||
| `tests/test_provider_endpoints_url_building.py` | 86 | 25 | services | provider | - | 25 collected tests |
|
||||
| `tests/test_research_utils.py` | 97 | 23 | services | research | - | 23 collected tests |
|
||||
| `tests/test_prompt_security.py` | 203 | 21 | security | security | - | 21 collected tests |
|
||||
| `tests/test_calendar_recurrence.py` | 338 | 19 | services | calendar | - | 338 lines |
|
||||
| `tests/test_calendar_recurrence.py` | 356 | 20 | services | calendar | - | 356 lines, 20 collected tests |
|
||||
| `tests/test_redos_llm_parsers.py` | 201 | 20 | services | llm | - | 20 collected tests |
|
||||
| `tests/test_llm_core_ollama.py` | 324 | 16 | services | llm | async/threading | 324 lines |
|
||||
| `tests/test_composer_arrow_up_recall_js.py` | 308 | 11 | js | js | filesystem, subprocess/script, ui/static | 308 lines |
|
||||
|
||||
## High-risk candidates to defer first
|
||||
|
||||
@@ -274,21 +307,21 @@ These files may still be split later, but not as the first implementation slice
|
||||
|
||||
| File | Lines | Collected tests | High-risk signals |
|
||||
|---|---:|---:|---|
|
||||
| `tests/test_model_routes.py` | 1778 | 139 | db/session, import-state, route/api |
|
||||
| `tests/test_security_regressions.py` | 1224 | 92 | db/session, import-state, route/api, security |
|
||||
| `tests/test_cookbook_helpers.py` | 912 | 65 | route/api |
|
||||
| `tests/test_shell_routes.py` | 481 | 63 | import-state, route/api |
|
||||
| `tests/test_model_routes.py` | 2099 | 163 | db/session, import-state, route/api |
|
||||
| `tests/test_security_regressions.py` | 1515 | 98 | db/session, import-state, route/api, security |
|
||||
| `tests/test_cookbook_helpers.py` | 958 | 69 | route/api |
|
||||
| `tests/test_shell_routes.py` | 538 | 69 | import-state, route/api |
|
||||
| `tests/test_endpoint_probing.py` | 497 | 61 | db/session, import-state, route/api |
|
||||
| `tests/test_pr_blocker_audit.py` | 964 | 58 | import-state, security |
|
||||
| `tests/test_agent_loop.py` | 469 | 52 | db/session, import-state |
|
||||
| `tests/test_run_focus.py` | 399 | 47 | security |
|
||||
| `tests/test_endpoint_probing.py` | 411 | 34 | db/session, import-state, route/api |
|
||||
| `tests/test_research_routes_path_confinement.py` | 565 | 58 | route/api, security |
|
||||
| `tests/test_run_focus.py` | 492 | 55 | security |
|
||||
| `tests/test_agent_loop.py` | 500 | 54 | db/session, import-state |
|
||||
| `tests/test_manage_mcp_command_allowlist.py` | 168 | 45 | db/session |
|
||||
| `tests/test_review_regressions.py` | 1429 | 41 | db/session, import-state, route/api |
|
||||
| `tests/test_chat_helpers.py` | 582 | 37 | route/api |
|
||||
| `tests/test_model_context.py` | 314 | 36 | db/session, import-state |
|
||||
| `tests/test_email_oauth.py` | 798 | 34 | db/session, route/api, security |
|
||||
| `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | db/session |
|
||||
| `tests/test_chat_helpers.py` | 264 | 31 | route/api |
|
||||
| `tests/test_model_context.py` | 251 | 30 | db/session, import-state |
|
||||
| `tests/test_upload_limits_centralized.py` | 110 | 29 | import-state |
|
||||
| `tests/test_email_oauth.py` | 580 | 28 | db/session, route/api, security |
|
||||
| `tests/test_review_regressions.py` | 930 | 26 | db/session, import-state, route/api |
|
||||
| `tests/test_rename_user_owner_sync.py` | 686 | 26 | db/session, import-state, route/api |
|
||||
|
||||
## Rules for future split PRs
|
||||
|
||||
@@ -313,7 +346,7 @@ Do not start a split PR from this planning issue alone if the file's boundaries
|
||||
This document was generated with:
|
||||
|
||||
```bash
|
||||
.venv/bin/python tests/tools/build_oversized_test_split_plan.py
|
||||
venv/bin/python tests/tools/build_oversized_test_split_plan.py
|
||||
```
|
||||
|
||||
## Freshness check
|
||||
@@ -321,6 +354,6 @@ This document was generated with:
|
||||
After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:
|
||||
|
||||
```bash
|
||||
.venv/bin/python tests/tools/build_oversized_test_split_plan.py
|
||||
venv/bin/python tests/tools/build_oversized_test_split_plan.py
|
||||
git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md
|
||||
```
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
"""Regression coverage for SMTP security saved before Google OAuth."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
_REPO = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def test_email_tab_oauth_connect_persists_selected_smtp_security():
|
||||
source = (_REPO / "static" / "js" / "settings.js").read_text(encoding="utf-8")
|
||||
start = source.index("el('eaf-oauth-btn').addEventListener")
|
||||
handler_body = source[start:source.index("if (!body.name)", start)]
|
||||
|
||||
assert "smtp_security: el('eaf-smtp-security').value" in handler_body
|
||||
assert "display_name: el('eaf-display-name').value.trim()" in handler_body
|
||||
@@ -1,19 +0,0 @@
|
||||
"""Regression coverage for the settings UI after Google OAuth redirects."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
_REPO = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def test_oauth_redirect_uses_the_module_local_settings_api():
|
||||
source = (_REPO / "static" / "js" / "settings.js").read_text(encoding="utf-8")
|
||||
handler = source[
|
||||
source.index("(function _handleOauthRedirect"):
|
||||
source.index("const settingsModule =")
|
||||
]
|
||||
|
||||
assert "open('integrations');" in handler
|
||||
assert "window.settingsModule" not in handler
|
||||
assert "window.__odysseusAppStarted" not in handler
|
||||
assert "document.addEventListener('DOMContentLoaded', _showResult, { once: true })" in handler
|
||||
@@ -1,86 +0,0 @@
|
||||
"""Regression coverage for issue-description label lifecycle events."""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
_REPO = Path(__file__).resolve().parent.parent
|
||||
_CHECKER = _REPO / ".github" / "scripts" / "check-issue-description.js"
|
||||
_WORKFLOW = _REPO / ".github" / "workflows" / "issue-description-check.yml"
|
||||
pytestmark = pytest.mark.skipif(not shutil.which("node"), reason="node not on PATH")
|
||||
|
||||
|
||||
def _run_closed_issue(action):
|
||||
harness = r"""
|
||||
const checkIssueDescription = require(process.argv[1]);
|
||||
const action = process.argv[2];
|
||||
const calls = [];
|
||||
const unexpected = (name) => async () => {
|
||||
throw new Error(`${name} should not be called for a closed issue`);
|
||||
};
|
||||
|
||||
const github = {
|
||||
rest: {
|
||||
issues: {
|
||||
removeLabel: async (params) => calls.push({ method: 'removeLabel', params }),
|
||||
getLabel: unexpected('getLabel'),
|
||||
addLabels: unexpected('addLabels'),
|
||||
listComments: unexpected('listComments'),
|
||||
createComment: unexpected('createComment'),
|
||||
updateComment: unexpected('updateComment'),
|
||||
deleteComment: unexpected('deleteComment'),
|
||||
},
|
||||
},
|
||||
};
|
||||
const context = {
|
||||
payload: {
|
||||
action,
|
||||
issue: { number: 42, state: 'closed', body: '', labels: [] },
|
||||
},
|
||||
repo: { owner: 'odysseus-dev', repo: 'odysseus' },
|
||||
};
|
||||
const core = {
|
||||
warning: unexpected('core.warning'),
|
||||
setFailed: unexpected('core.setFailed'),
|
||||
};
|
||||
|
||||
checkIssueDescription({ github, context, core })
|
||||
.then(() => process.stdout.write(JSON.stringify(calls)))
|
||||
.catch((error) => {
|
||||
console.error(error);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
"""
|
||||
proc = subprocess.run(
|
||||
["node", "-e", harness, str(_CHECKER), action],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd=str(_REPO),
|
||||
timeout=30,
|
||||
)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
return json.loads(proc.stdout)
|
||||
|
||||
|
||||
def test_workflow_handles_issue_closures():
|
||||
workflow = _WORKFLOW.read_text()
|
||||
assert "types: [opened, edited, reopened, closed]" in workflow
|
||||
|
||||
|
||||
@pytest.mark.parametrize("action", ["closed", "edited"])
|
||||
def test_closed_issue_only_drops_ready_for_review(action):
|
||||
assert _run_closed_issue(action) == [
|
||||
{
|
||||
"method": "removeLabel",
|
||||
"params": {
|
||||
"owner": "odysseus-dev",
|
||||
"repo": "odysseus",
|
||||
"issue_number": 42,
|
||||
"name": "ready for review",
|
||||
},
|
||||
}
|
||||
]
|
||||
@@ -29,13 +29,6 @@ from src.llm_core import _anthropic_rejects_temperature, _build_anthropic_payloa
|
||||
"anthropic/claude-opus-4-7", # tolerate a provider-prefixed id
|
||||
"claude-opus-4-10", # future minor still >= 4.7
|
||||
"claude-opus-5-0", # future major
|
||||
# Major-only ids: a missing minor reads as `.0`, so these are >= 4.7 too
|
||||
# (issue #5753). Before the fix the version pattern required a minor, so
|
||||
# these fell through to "accepts temperature" and every call 400'd.
|
||||
"claude-opus-5",
|
||||
"claude-opus-5-20260101", # major-only + dated snapshot
|
||||
"anthropic/claude-opus-5", # major-only behind a provider prefix
|
||||
"claude-opus-6", # future major-only
|
||||
],
|
||||
)
|
||||
def test_opus_47_plus_rejects_temperature(model):
|
||||
@@ -55,10 +48,7 @@ def test_opus_47_plus_rejects_temperature(model):
|
||||
"claude-opus-4-6-20251201", # dated 4.6 snapshot — older, still keeps temperature
|
||||
"claude-sonnet-4-6",
|
||||
"claude-3-5-sonnet",
|
||||
"claude-3-opus-20240229", # legacy Claude 3 Opus — date directly after
|
||||
# "opus-", so the major must not swallow it as version 20240229 (that is
|
||||
# what makes capping the major at 1-2 digits necessary once the minor
|
||||
# became optional in #5753).
|
||||
"claude-3-opus-20240229", # legacy Claude 3 Opus — no opus-N-M pattern, kept
|
||||
"claude-haiku-4-5",
|
||||
"claude-x",
|
||||
"octopus-4-8", # "opus" only as a substring of another word — must not match
|
||||
@@ -97,20 +87,6 @@ def test_payload_keeps_temperature_for_older_models():
|
||||
assert _payload("claude-3-5-sonnet", 1.2)["temperature"] == 1.0
|
||||
|
||||
|
||||
def test_payload_omits_temperature_for_major_only_opus_5():
|
||||
# Issue #5753: the scheduled-task path calls stream_agent_loop() without a
|
||||
# temperature and inherits its 0.3 default, so `claude-opus-5` 400'd on every
|
||||
# run and surfaced as "the model returned an empty response". Interactive chat
|
||||
# leaves temperature None and never hit it.
|
||||
assert "temperature" not in _payload("claude-opus-5", 0.3)
|
||||
|
||||
|
||||
def test_payload_keeps_temperature_for_legacy_claude_3_opus():
|
||||
# Guards the major-digit cap: `opus-20240229` must not parse as version
|
||||
# 20240229, or Claude 3 Opus would silently lose the caller's temperature.
|
||||
assert _payload("claude-3-opus-20240229", 0.5)["temperature"] == 0.5
|
||||
|
||||
|
||||
def test_payload_keeps_temperature_for_dated_opus_4_0():
|
||||
# Anthropic's dated id for Opus 4.0 (claude-opus-4-20250514) is in this repo's
|
||||
# ANTHROPIC_MODELS list. The date must not be misread as a >= 4.7 minor, or the
|
||||
|
||||
@@ -214,50 +214,6 @@ def test_inline_code_content_is_html_escaped(node_available):
|
||||
assert "<b>" not in html
|
||||
|
||||
|
||||
def test_fenced_code_keeps_dollar_ampersand(node_available):
|
||||
# Issue #5663: the block-restore pass used a string replacement, so `$&` in a
|
||||
# restored block was read as "the matched text" and re-inserted the
|
||||
# placeholder. `perl -pe 's/world/$& again/'` rendered as
|
||||
# "s/world/___CODE_BLOCK_0___amp; again/" — the trailing "amp;" is the orphan
|
||||
# left behind after `$&` consumed the `$&` of the escaped `$&`.
|
||||
html = _run_markdown_case(
|
||||
"```sh\necho \"hello world\" | perl -pe 's/world/$& again/'\n```"
|
||||
)
|
||||
|
||||
assert "___CODE_BLOCK_" not in html
|
||||
assert "s/world/$& again/" in html
|
||||
assert "amp; again" not in html.replace("$& again", "")
|
||||
|
||||
|
||||
def test_fenced_code_keeps_dollar_backtick_and_quote(node_available):
|
||||
# `` $` `` and `$'` splice the text before/after the placeholder into the
|
||||
# block. Unlike `$&` these leave no placeholder behind — the characters just
|
||||
# vanish — so assert the content survives verbatim.
|
||||
html = _run_markdown_case("```sh\nsed \"s/$`/x/\" && sed \"s/$'/y/\"\n```")
|
||||
|
||||
assert "___CODE_BLOCK_" not in html
|
||||
assert "s/$`/x/" in html
|
||||
assert "s/$'/y/" in html
|
||||
|
||||
|
||||
def test_fenced_code_keeps_double_dollar(node_available):
|
||||
# `$$` collapsed to a single `$` in the restored block.
|
||||
html = _run_markdown_case('```sh\necho "$$USD and $$"\n```')
|
||||
|
||||
assert "$$USD and $$" in html
|
||||
|
||||
|
||||
def test_mermaid_block_keeps_dollar_ampersand(node_available):
|
||||
# The mermaid restore site had the same hazard: a node label containing `$&`
|
||||
# re-inserted the ___MERMAID_BLOCK_n___ placeholder into the diagram source,
|
||||
# which then fails to parse. The math and allowed-HTML sites are fixed the
|
||||
# same way; they need KaTeX/sanitizer conditions this harness doesn't set up.
|
||||
html = _run_markdown_case('```mermaid\ngraph TD; A["$&"] --> B;\n```')
|
||||
|
||||
assert "___MERMAID_BLOCK_" not in html
|
||||
assert "$&" in html
|
||||
|
||||
|
||||
def test_currency_dollar_amounts_are_not_rendered_as_math(node_available):
|
||||
# "$5 to $10" used to pair the two dollar signs as inline-math delimiters
|
||||
# and render "5 to" through KaTeX. Pandoc-style rules now reject it: the
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
"""Regression coverage for the built-in MCP servers' SDK compatibility line."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
REQUIREMENTS = Path(__file__).resolve().parents[1] / "requirements.txt"
|
||||
|
||||
|
||||
def test_mcp_requirement_excludes_breaking_v2_sdk():
|
||||
requirements = [
|
||||
line.split("#", 1)[0].strip().replace(" ", "")
|
||||
for line in REQUIREMENTS.read_text(encoding="utf-8").splitlines()
|
||||
]
|
||||
|
||||
assert "mcp<2" in requirements
|
||||
@@ -1,8 +1,8 @@
|
||||
"""Provider endpoint URL-building tests.
|
||||
|
||||
Covers ``build_chat_url`` and ``build_models_url`` for every provider named in
|
||||
ROADMAP.md: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, DeepSeek, Ollama
|
||||
(local + cloud).
|
||||
Covers ``build_chat_url`` and ``build_models_url`` for the provider families
|
||||
exercised by this module: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI,
|
||||
DeepSeek, and Ollama (local and cloud).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
"""Regression test for the search route shim (slice 2j, #4082/#4071)."""
|
||||
|
||||
import importlib
|
||||
|
||||
import routes.search_routes as _shim_search # noqa: F401
|
||||
|
||||
|
||||
def test_legacy_and_canonical_search_module_are_same_object():
|
||||
legacy = importlib.import_module("routes.search_routes")
|
||||
canonical = importlib.import_module("routes.search.search_routes")
|
||||
assert legacy is canonical
|
||||
@@ -1,58 +0,0 @@
|
||||
"""Regression for issue #5697 — skill timestamps must not use ``datetime.utcnow()``.
|
||||
|
||||
``_now_iso()`` builds the ``created`` value in skill frontmatter. ``utcnow()``
|
||||
returns a *naive* datetime and has been deprecated since Python 3.12, scheduled
|
||||
for removal. The replacement must stay timezone-aware while keeping the
|
||||
serialized ``YYYY-MM-DDTHH:MM:SSZ`` shape, so skill files written by older
|
||||
versions keep parsing.
|
||||
|
||||
The UTC check matters on its own: a bare ``datetime.now()`` also produces the
|
||||
right shape, but emits local wall time, which would silently backdate or
|
||||
postdate skills for every user outside UTC.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import warnings
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from services.memory.skill_format import _now_iso
|
||||
|
||||
_ISO_Z = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$")
|
||||
|
||||
|
||||
def test_now_iso_keeps_serialized_shape():
|
||||
assert _ISO_Z.match(_now_iso())
|
||||
|
||||
|
||||
def test_now_iso_emits_no_deprecation_warning():
|
||||
with warnings.catch_warnings(record=True) as caught:
|
||||
warnings.simplefilter("always")
|
||||
_now_iso()
|
||||
assert not [w for w in caught if issubclass(w.category, DeprecationWarning)]
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not hasattr(time, "tzset"),
|
||||
reason="time.tzset is unavailable on this platform",
|
||||
)
|
||||
def test_now_iso_is_utc_not_local_time():
|
||||
"""Pin UTC under a non-UTC local timezone, where the two visibly diverge."""
|
||||
original_tz = os.environ.get("TZ")
|
||||
os.environ["TZ"] = "Asia/Amman" # UTC+3, never UTC
|
||||
time.tzset()
|
||||
try:
|
||||
emitted = datetime.strptime(_now_iso(), "%Y-%m-%dT%H:%M:%SZ").replace(
|
||||
tzinfo=timezone.utc
|
||||
)
|
||||
drift = abs((emitted - datetime.now(timezone.utc)).total_seconds())
|
||||
assert drift < 60, f"timestamp is {drift}s off UTC — local time leaked in"
|
||||
finally:
|
||||
if original_tz is None:
|
||||
os.environ.pop("TZ", None)
|
||||
else:
|
||||
os.environ["TZ"] = original_tz
|
||||
time.tzset()
|
||||
@@ -1,97 +0,0 @@
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
import pytest
|
||||
|
||||
# Adjust the import path if your file is directly in ./services instead of ./services/tts
|
||||
from services.tts.tts_service import TTSService
|
||||
|
||||
def test_cache_under_limit(tmp_path, monkeypatch):
|
||||
"""Test that writing a file under the size limit does not trigger eviction."""
|
||||
# Set a tiny limit: 100 bytes
|
||||
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "100")
|
||||
|
||||
# Initialize service with pytest's temporary directory
|
||||
service = TTSService(cache_dir=str(tmp_path))
|
||||
|
||||
# Write a 40-byte file (under the 100-byte limit)
|
||||
service._put_cache("test_key", b"x" * 40)
|
||||
|
||||
# Verify the file was written and nothing was deleted
|
||||
files = list(tmp_path.glob("*.*"))
|
||||
assert len(files) == 1
|
||||
assert sum(f.stat().st_size for f in files) == 40
|
||||
|
||||
def test_cache_exceeds_limit_triggers_eviction(tmp_path, monkeypatch):
|
||||
"""Test that exceeding the limit evicts the oldest files down to 80% capacity."""
|
||||
# Set limit to 100 bytes. 80% target capacity will be 80 bytes.
|
||||
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "100")
|
||||
service = TTSService(cache_dir=str(tmp_path))
|
||||
|
||||
# 1. Setup: Manually create two older files (40 bytes each)
|
||||
file1 = tmp_path / "oldest.wav"
|
||||
file2 = tmp_path / "middle.wav"
|
||||
|
||||
file1.write_bytes(b"a" * 40)
|
||||
file2.write_bytes(b"b" * 40)
|
||||
|
||||
# Spoof timestamps so file1 is explicitly older than file2
|
||||
now = time.time()
|
||||
os.utime(file1, (now - 100, now - 100)) # 100 seconds ago
|
||||
os.utime(file2, (now - 50, now - 50)) # 50 seconds ago
|
||||
|
||||
# 2. Action: Write a 3rd file using the service method (40 bytes)
|
||||
# Total cache is now 120 bytes, which exceeds 100.
|
||||
# It should delete oldest (file1) to drop to 80 bytes (which matches the 80% target).
|
||||
service._put_cache("newest", b"c" * 40)
|
||||
|
||||
# 3. Assertions
|
||||
# The newest file should exist (saved as .wav because it lacks MP3 magic bytes)
|
||||
newest_file = tmp_path / "newest.wav"
|
||||
|
||||
assert not file1.exists(), "The oldest file should have been evicted."
|
||||
assert file2.exists(), "The middle file should still exist."
|
||||
assert newest_file.exists(), "The newest file should have been saved."
|
||||
|
||||
# Verify the final directory size is <= 80 bytes
|
||||
total_size = sum(f.stat().st_size for f in tmp_path.glob("*.*"))
|
||||
assert total_size <= 80
|
||||
|
||||
def test_cache_limit_disabled(tmp_path, monkeypatch):
|
||||
"""Test that setting max bytes to 0 disables eviction."""
|
||||
monkeypatch.setenv("ODYSSEUS_TTS_CACHE_MAX_BYTES", "0")
|
||||
service = TTSService(cache_dir=str(tmp_path))
|
||||
|
||||
# Write 3 large files that would normally trigger eviction
|
||||
service._put_cache("file1", b"x" * 1000)
|
||||
service._put_cache("file2", b"x" * 1000)
|
||||
service._put_cache("file3", b"x" * 1000)
|
||||
|
||||
# Ensure nothing was deleted
|
||||
files = list(tmp_path.glob("*.*"))
|
||||
assert len(files) == 3
|
||||
assert sum(f.stat().st_size for f in files) == 3000
|
||||
|
||||
def test_cache_eviction_handles_unlink_error_gracefully(tmp_path, monkeypatch):
|
||||
"""Test that if unlinking a file fails, _put_cache still succeeds without raising."""
|
||||
service = TTSService(cache_dir=str(tmp_path))
|
||||
service.max_cache_bytes = 50
|
||||
|
||||
# Create a file to evict
|
||||
old_file = tmp_path / "old.wav"
|
||||
old_file.write_bytes(b"x" * 40)
|
||||
|
||||
# Monkeypatch unlink on Path objects to simulate a PermissionError / file-lock failure
|
||||
def mock_unlink(self_path):
|
||||
raise OSError("Permission denied / file locked")
|
||||
|
||||
monkeypatch.setattr(Path, "unlink", mock_unlink)
|
||||
|
||||
# Writing a new file triggers eviction which encounters the mocked unlink error
|
||||
try:
|
||||
service._put_cache("new_key", b"y" * 40)
|
||||
except Exception as e:
|
||||
pytest.fail(f"_put_cache raised an exception during failed eviction: {e}")
|
||||
|
||||
# The new file should still be written successfully
|
||||
assert (tmp_path / "new_key.wav").exists()
|
||||
@@ -441,6 +441,14 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
|
||||
lines = [
|
||||
"# Oversized Test File Split Plan",
|
||||
"",
|
||||
"> [!NOTE]",
|
||||
"> This is generated planning material based on the current test tree.",
|
||||
">",
|
||||
"> It is non-canonical and disposable. Durable testing policy belongs in",
|
||||
"> `tests/TESTING_STANDARD.md`; active split selection and prioritization",
|
||||
"> belong in approved issues or the project tracker.",
|
||||
"> It does not define runtime behaviour or approve any test-file split.",
|
||||
"",
|
||||
"## Purpose",
|
||||
"",
|
||||
"This document plans future oversized test-file splits using current repo data.",
|
||||
@@ -508,7 +516,7 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
|
||||
"This document was generated with:",
|
||||
"",
|
||||
"```bash",
|
||||
".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
|
||||
"venv/bin/python tests/tools/build_oversized_test_split_plan.py",
|
||||
"```",
|
||||
"",
|
||||
"## Freshness check",
|
||||
@@ -516,7 +524,7 @@ def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
|
||||
"After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:",
|
||||
"",
|
||||
"```bash",
|
||||
".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
|
||||
"venv/bin/python tests/tools/build_oversized_test_split_plan.py",
|
||||
"git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md",
|
||||
"```",
|
||||
"",
|
||||
|
||||
Reference in New Issue
Block a user