test: refresh oversized split plan

test: add oversized test split plan
fix(companion): require chat scope for model inventory (#4319 )
2026-06-17 18:25:26 -04:00 · 2026-06-16 02:14:25 +01:00 · 2026-06-16 02:14:13 +01:00 · 2026-06-16 01:15:05 +02:00 · 2026-06-16 01:00:40 +02:00 · 2026-06-16 00:41:45 +02:00
82 changed files with 4243 additions and 740 deletions
@@ -1,61 +0,0 @@
 # CodeQL code scanning
 #
 # Purpose: GitHub's own static analysis engine reads the application source
 # (Python backend + the JavaScript frontend) and looks for real
 # vulnerabilities -- SQL/command injection, path traversal, auth mistakes,
 # unsafe deserialization. Findings appear in the repo's Security tab. This is
 # the deepest check in the suite and the most valuable for a high-profile
 # target.
 #
 # It runs on every push to main and on a weekly schedule (to catch newly
 # disclosed query patterns against unchanged code). It deliberately does NOT
 # run on pull requests: most PRs here come from forks, whose read-only token
 # cannot publish results, which would produce confusing failures. To scan pull
 # requests too, a maintainer can instead enable CodeQL "default setup" in
 # Settings -> Security -> Code scanning (one toggle, no file needed) -- see
 # docs/security-ci.md.
 name: CodeQL
 on:
  push:
    branches: [main]
  schedule:
    # Weekly, Monday 06:00 UTC.
    - cron: '0 6 * * 1'
  workflow_dispatch:
 permissions: {}
 concurrency:
  group: codeql-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 jobs:
  analyze:
    name: Analyze (${{ matrix.language }})
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write  # publish results to the Security tab
    strategy:
      fail-fast: false
      matrix:
        # Both are interpreted, so CodeQL needs no build step (build-mode none).
        language: [python, javascript-typescript]
    steps:
      - name: Checkout repository
        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10  # v6.0.3
        with:
          persist-credentials: false
      - name: Initialize CodeQL
        uses: github/codeql-action/init@8aad20d150bbac5944a9f9d289da16a4b0d87c1e  # v4.36.2
        with:
          languages: ${{ matrix.language }}
          build-mode: none
      - name: Perform CodeQL analysis
        uses: github/codeql-action/analyze@8aad20d150bbac5944a9f9d289da16a4b0d87c1e  # v4.36.2
        with:
          category: "/language:${{ matrix.language }}"
@@ -1,476 +1,65 @@
-# Odysseus
+<p align="center">
  <img src="docs/odysseus-wordmark.png" alt="Odysseus" width="280">
 </p>
-> **Branch note:** `dev` is the default branch and contains the latest development changes, but it may be unstable. For the more stable curated branch, use [`main`](https://github.com/pewdiepie-archdaemon/odysseus/tree/main).
+<p align="center">
  A self-hosted AI workspace for chat, agents, research, documents, email, notes, calendar, and local model workflows.
 </p>
-```
+<p align="center">
-───────────────────────────────────────────────
+  <a href="#quick-start">Quick Start</a> ·
- ⊹ ࣪ ˖ ૮( ˶ᵔ ᵕ ᵔ˶ )っ  Odysseus vers. 1.0
+  <a href="docs/setup.md">Setup Guide</a> ·
-───────────────────────────────────────────────
+  <a href="CONTRIBUTING.md">Contributing</a> ·
-```
+  <a href="ROADMAP.md">Roadmap</a>
 </p>
-![Odysseus](docs/odysseus.jpg)
+<p align="center">
  <a href="https://repology.org/project/odysseus-ai/versions"><img src="https://repology.org/badge/vertical-allrepos/odysseus-ai.svg" alt="Packaging status"></a>
 </p>
-A self-hosted AI workspace -- meant to be the self-hosted version of the UI experience you get from ChatGPT and Claude. But with more jank and fun. Running on your own hardware, with your own data -- local-first, privacy-first, and no trojan.
+<p align="center">
  <img src="docs/odysseus.jpg" alt="Odysseus interface">
 </p>
-[![Packaging status](https://repology.org/badge/vertical-allrepos/odysseus-ai.svg)](https://repology.org/project/odysseus-ai/versions)
+---
 ## Features
  - **Chat** -- chat with any local model or API; adding them is super simple.<br>　<sub>vLLM · llama.cpp · Ollama · OpenRouter · OpenAI · GitHub Copilot</sub>
  - **Agent** -- hand it tools and let it run the whole task itself.<br>　<sub>built on [opencode](https://github.com/anomalyco/opencode) · MCP · web · files · shell · skills · memory</sub>
  - **Cookbook** -- Scans your hardware, recommends models, click to download and serve.. easy!<br>　<sub>built on [llmfit](https://github.com/AlexsJones/llmfit) · VRAM-aware · GGUF / FP8 / AWQ · fit scoring · vLLM / llama.cpp serving</sub>
  - **Deep Research** -- multi-step runs that gather, read, and synthesize sources into a nice visual report.<br>　<sub>adapted from [Tongyi DeepResearch](https://github.com/Alibaba-NLP/DeepResearch)</sub>
  - **Compare** -- a fun tool to compare models side by side. Test completely blind, no bias!<br>　<sub>multi-model · blind test · synthesis</sub>
  - **Documents** -- YOU write the text, AI is there to assist, not the opposite.<br>　<sub>multi-tab editor · markdown · HTML · CSV · syntax highlighting · AI edits · suggestions</sub>
  - **Memory / Skills** -- Persistent memory and skills, your agent evolves over time as it better understands you and your tasks!<br>　<sub>ChromaDB · fastembed (ONNX) · vector + keyword retrieval · import/export</sub>
  - **Email** -- IMAP/SMTP inbox with AI triage built in: urgency reminders, auto-tag, auto-summary, auto-reply drafts, auto-spam.<br>　<sub>IMAP · SMTP · per-account routing · CalDAV-aware</sub>
  - **Notes & Tasks** -- Quick notes with reminders, a todo list, and scheduled tasks the agent can act on.<br>　<sub>note pings · checklist · cron-style tasks · ntfy / browser / email channels</sub>
  - **Calendar** -- Local-first calendar with CalDAV sync to Radicale / Nextcloud / Apple / Fastmail.<br>　<sub>CalDAV pull · .ics import/export · per-calendar colors · agent-aware</sub>
  - **Works on mobile** -- looks and runs great on your phone, not just desktop.<br>　<sub>responsive · installable (PWA) · touch gestures</sub>
  - **Extras** -- more to explore, happy if you give it a go!<br>　<sub>image editor · theme editor · file uploads (vision + PDF) · web search · presets · sessions · 2FA</sub>
 ## Demo
 A full, hover-to-play tour lives on the landing page (`docs/index.html`).
 <details>
 <summary>Screenshots / clips</summary>
 ### Chat & Agents
 ![Chat & Agents](docs/chat.gif)
 ### Deep Research
 ![Deep Research](docs/research.gif)
 ### Compare
 ![Compare](docs/compare.gif)
 ### Documents
 ![Documents](docs/document.gif)
 ### Notes & Tasks
 ![Notes & Tasks](docs/notes.gif)
 </details>
 ## Quick Start
-Defaults work out of the box: clone, run, then configure models/search/email
+> `dev` is the default branch and gets the newest changes first. Use [`main`](https://github.com/pewdiepie-archdaemon/odysseus/tree/main) if you want the more curated branch.
 inside **Settings**. Only edit `.env` for deployment-level overrides like
 `APP_BIND`, `APP_PORT`, `AUTH_ENABLED`, `DATABASE_URL`, or a pre-seeded admin password.
 On first setup, Odysseus creates an admin account (`admin` unless
 `ODYSSEUS_ADMIN_USER` is set) and prints a temporary password in the terminal.
 For Docker installs, the same line is in `docker compose logs odysseus`.
 Use that for the first login, then change it in **Settings**.
 Contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, testing, and
 pull request guidelines.
 ### Docker (recommended)
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
-cp .env.example .env       # optional, but recommended for explicit defaults
+cp .env.example .env
 docker compose up -d --build
 ```
 To include optional extras in the image (PDF viewer, Office extraction; includes AGPL PyMuPDF), build with `docker compose build --build-arg INSTALL_OPTIONAL=true` before `up`.
-Open `http://localhost:7000` when the containers are healthy. Docker Compose
+Open `http://localhost:7000` when the containers are healthy. The first admin password is printed in `docker compose logs odysseus`.
 binds the web UI to `127.0.0.1` by default. If the port is taken, set
 `APP_PORT=7001` in `.env` and recreate the container. Set `APP_BIND=0.0.0.0`
 only when you intentionally want LAN/reverse-proxy access.
-> **On Apple Silicon (M-series) Macs:** Docker can't reach the Metal GPU, so
+Native installs, GPU notes, Windows/macOS instructions, HTTPS, and configuration live in the [setup guide](docs/setup.md).
 > Cookbook serves local models on CPU only. For GPU-accelerated model serving,
 > run natively instead — see [Apple Silicon](#apple-silicon) below.
-### Native Linux / macOS
+## Features
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 python3 -m venv venv
 source venv/bin/activate
 pip install -r requirements.txt
 python setup.py
 python -m uvicorn app:app --host 127.0.0.1 --port 7000
 ```
 Requirements: Python 3.11+. Cookbook also needs `tmux` for background model
 downloads and serves. The app itself is lightweight; local model serving is the
 heavy part and depends on the model, runtime, GPU, and VRAM, so small hosts can
 connect to API or remote model servers instead. Use `--host 0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
-### Apple Silicon
+- **Chat + Agents** — local/API models, tools, MCP, files, shell, skills, and memory.
-Docker on macOS cannot use the Metal GPU. For GPU-accelerated Cookbook on an
+- **Cookbook** — hardware-aware model recommendations, downloads, and serving.
-M-series Mac, run Odysseus natively:
+- **Deep Research** — multi-step web research with source reading and report generation.
 - **Compare** — blind side-by-side model testing and synthesis.
 - **Documents** — writing-first editor with AI edits, suggestions, Markdown, HTML, CSV, and syntax highlighting.
 - **Email** — IMAP/SMTP inbox with triage, tags, summaries, reminders, and reply drafts.
 - **Notes, Tasks + Calendar** — reminders, todos, scheduled agent tasks, and CalDAV sync.
 - **Extras** — gallery/image editor, themes, uploads, web search, presets, sessions, and 2FA.
-```bash
+## Demo
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 ./start-macos.sh
 ```
-It launches at `http://127.0.0.1:7860`. To expose it to your phone over a trusted LAN/VPN such as Tailscale, bind all interfaces:
+A full hover-to-play tour lives on the landing page: [`docs/index.html`](docs/index.html).
 ```bash
 ODYSSEUS_HOST=0.0.0.0 ./start-macos.sh
 # then open http://<tailscale-ip>:7860
 ```
 The script also reads `.env` at startup, so `APP_BIND=0.0.0.0` and `APP_PORT`
 set there are picked up automatically without a command-line override each run.
 Keep `AUTH_ENABLED=true` (the default) before binding outside loopback. Do not
 expose this port directly to the public internet. To build a clickable app wrapper:
 ```bash
 ./build-macos-app.sh
 ```
 <details>
 <summary>Cookbook, GPU, Ollama, and troubleshooting notes</summary>
 **Docker bundled services.** Compose starts Odysseus, ChromaDB, SearXNG, and
 ntfy. Odysseus and the bundled service ports bind to `127.0.0.1` by default, so
 they are reachable from the host but not exposed to your LAN/public internet
 unless you opt in.
 **Cookbook storage in Docker.** Downloads live in `./data/huggingface`
 (`~/.cache/huggingface` in the container). Cookbook-installed Python CLIs and
 serve engines live in `./data/local` (`~/.local` in the container), so they
 survive container recreation.
 **Remote servers.** In **Cookbook -> Settings -> Servers**, generate the
 Odysseus SSH key and add the public key to the remote server's
 `~/.ssh/authorized_keys`. From the host you can also run:
 ```bash
 ssh-copy-id -i data/ssh/id_ed25519.pub user@server
 ```
 **Docker GPU overlays.** CPU-only users can skip this section. Cookbook can
 only detect GPUs that Docker exposes to the container — if the host runtime or
 device passthrough is not configured, Cookbook sees the iGPU, another card, or
 CPU instead of your intended GPU.
 For NVIDIA, `scripts/check-docker-gpu.sh` diagnoses GPU passthrough and can
 optionally install the host runtime or update `.env`.
 ```bash
 # Read-only diagnostic (default — installs nothing, never edits .env):
 scripts/check-docker-gpu.sh
 # Print OS-specific install commands without running them:
 scripts/check-docker-gpu.sh --print-install-commands
 # Install NVIDIA Container Toolkit on Ubuntu/Debian (requires sudo):
 scripts/check-docker-gpu.sh --install-nvidia-toolkit
 # Write COMPOSE_FILE to .env (only when GPU passthrough is confirmed working):
 scripts/check-docker-gpu.sh --enable-nvidia-overlay
 # Full assisted setup — install toolkit, then enable overlay if passthrough works:
 scripts/check-docker-gpu.sh --install-nvidia-toolkit --enable-nvidia-overlay
 ```
 Safety notes:
 - The app never installs host GPU runtime automatically.
 - The app never edits `.env` automatically.
 - `.env` is only modified when `--enable-nvidia-overlay` is explicitly passed,
  and only after GPU passthrough succeeds. `--yes` skips prompts but does not
  bypass the passthrough gate.
 - `.env.bak.*` backups created by `--enable-nvidia-overlay` are ignored by
  Git and the Docker build context.
 To enable manually without the script, add this to `.env`:
 ```bash
 COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
 ```
 **AMD / ROCm.** AMD setup is read-only diagnostic plus manual `.env` edit. Run:
 ```bash
 scripts/check-docker-amd-gpu.sh
 ```
 Then add the reported values to `.env`, replacing `RENDER_GID` with your host's
 numeric render group id:
 ```bash
 COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
 RENDER_GID=989
 ```
 For NVIDIA/AMD GPU support, also read the comments in the selected overlay file: docker/gpu.nvidia.yml or docker/gpu.amd.yml.
 **Stack-management UIs (Portainer, Coolify, Dockhand, etc.).** These tools
 often accept only a single Compose file and do not reliably honor `COMPOSE_FILE`
 or multiple `-f` overlays. CLI users should keep using the `COMPOSE_FILE`
 overlay workflow above. For stack UIs, point the stack at one of the standalone
 files instead, which bundle the base stack plus the GPU settings:
 - `docker-compose.gpu-nvidia.yml` — still requires the NVIDIA Container Toolkit
  on the host.
 - `docker-compose.gpu-amd.yml` — still requires host ROCm/kfd/DRI setup, the
  `video`/`render` group membership, and `RENDER_GID` when needed.
 The base `docker-compose.yml` plus the `docker/gpu.*.yml` overlays remain the
 source of truth; the standalone files mirror them for single-file deployments.
 Verify after enabling either overlay:
 ```bash
 docker compose exec odysseus nvidia-smi -L   # NVIDIA
 docker compose exec odysseus sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls -l /dev/kfd /dev/dri/renderD*'  # AMD
 ```
 > **GPU passthrough ≠ llama.cpp CUDA.** `nvidia-smi` passing inside the
 > container confirms Docker GPU access, but llama.cpp also needs `cudart` and
 > the CUDA Toolkit at runtime. If Cookbook logs show `Unable to find cudart
 > library`, `Could NOT find CUDAToolkit`, `CUDA Toolkit not found`, or
 > tensors/layers assigned to CPU, that is a Cookbook/llama.cpp build issue —
 > not a Docker passthrough failure. Reinstall the serve engine via
 > **Cookbook → Dependencies** to get a CUDA-enabled build.
 >
 > The same split applies to AMD/ROCm: seeing `/dev/kfd` and `/dev/dri` inside
 > the container confirms device passthrough, not ROCm userspace or a
 > ROCm-enabled vLLM/llama.cpp build. `rocm-smi` and `rocminfo` are not expected
 > inside the slim Odysseus image.
 **Ollama with Docker.** If Ollama runs on the host, add this endpoint in
 Settings:
 ```text
 http://host.docker.internal:11434/v1
 ```
 Ollama must listen outside its own loopback interface:
 ```bash
 OLLAMA_HOST=0.0.0.0:11434 ollama serve
 ```
 This connects Odysseus in Docker to an Ollama server that is already running on
 your host machine; it does not start Ollama inside the container.
 `host.docker.internal` is Docker's hostname for the host machine from inside the
 container. Cookbook **Serve** is a separate workflow for serving downloaded
 models through Odysseus/llama.cpp, so Windows users with an existing Ollama
 install usually only need to add the endpoint in Settings.
 **Useful checks.**
 ```bash
 docker compose ps
 docker compose logs --tail=120 odysseus
 docker compose logs odysseus | grep -E 'ChromaDB|MemoryVectorStore|DEGRADED'
 ```
 **macOS details.** `start-macos.sh` installs Homebrew deps, creates the venv,
 runs setup, and starts uvicorn on port `7860` because AirPlay often holds
 `7000`. It uses llama.cpp/Ollama for Metal. vLLM/SGLang are CUDA/ROCm-only and
 do not run on macOS. MLX-only models are not served by Odysseus.
 </details>
 ### Native Windows
 **One-command launcher** (creates the venv, installs deps, runs setup, starts the
 server; safe to re-run):
 ```powershell
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 powershell -ExecutionPolicy Bypass -File .\launch-windows.ps1
 ```
 Or do it by hand:
 ```powershell
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 py -3.11 -m venv venv
 venv\Scripts\Activate.ps1
 pip install -r requirements.txt
 python setup.py
 python -m uvicorn app:app --host 127.0.0.1 --port 7000
 ```
 If `python` points at an older interpreter, use `py -3.12` (or another installed
 3.11+ version) for the venv step.
 **Requirements:** Python 3.11+. The core app (chat, agent, memory, documents,
 email, calendar, deep research) runs fully native. For full **Cookbook** background
 model downloads and the agent shell tool, also install
 [Git for Windows](https://git-scm.com/download/win) (provides `bash.exe`).
 Local GPU *serving* of vLLM/SGLang needs Linux/WSL2; for a local model on Windows,
 [Ollama](https://ollama.com/download) is the easiest path — point Odysseus at
 `http://localhost:11434/v1` in Settings.
 Open `http://localhost:7000`, log in with the generated admin password,
 and configure everything else inside **Settings**.
 ## Troubleshooting & Advanced Setup
 ### `chromadb-client` conflicts with embedded ChromaDB
 If `chromadb-client` (the lightweight HTTP-only package) is installed alongside the full `chromadb` package, Odysseus starts but ChromaDB silently falls back to HTTP-only mode and fails.
 **Fix:** uninstall `chromadb-client` and force-reinstall the full package:
 ```bash
 ./venv/bin/pip uninstall chromadb-client -y
 ./venv/bin/pip install --force-reinstall chromadb
 ```
 ### HTTPS + LAN/Tailscale exposure
 To expose Odysseus on a local network or Tailscale with HTTPS:
 1. Change the bind address to `0.0.0.0` in `.env` (`APP_BIND=0.0.0.0` or `ODYSSEUS_HOST=0.0.0.0`).
 2. Generate a locally-trusted cert for your LAN/Tailscale IPs using [mkcert](https://github.com/FiloSottile/mkcert):
   ```bash
   mkcert -install
   mkcert -cert-file cert.pem -key-file key.pem 192.168.1.100 tailscale-ip
   ```
 3. Run `uvicorn` with the generated certs:
   ```bash
   python -m uvicorn app:app --host 0.0.0.0 --port 7000 --ssl-certfile=cert.pem --ssl-keyfile=key.pem
   ```
 4. Install the `mkcert` CA on any other device you want to access Odysseus from (e.g., for iOS, email the `rootCA.pem` to yourself, install the profile, and trust it in Certificate Trust Settings).
 ### Optional Dependencies
 `requirements-optional.txt` contains packages that unlock extra features. It is not installed by default.
 | Package | Feature unlocked |
 |---------|-----------------|
 | `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
 | `ddgs` | DuckDuckGo as a search provider option. |
 | `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
 | `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
 ### Faster, reproducible installs with uv (optional)
 [uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
 venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
 ```bash
 uv venv venv --python 3.13
 uv pip install -r requirements.txt
 # then continue as usual: python setup.py, uvicorn, ...
 ```
 `requirements.txt` is intentionally unpinned, so two installs at different times can produce different package versions. If you want a reproducible environment (e.g. across your own machines, or to roll back after a bad upgrade), snapshot and restore exact versions with:
 ```bash
 uv pip compile requirements.txt -o requirements.lock   # snapshot current resolution
 uv pip sync requirements.lock                          # reproduce it exactly later
 ```
 `requirements.lock` is gitignored and platform-specific (compile it on the OS you deploy to). Regenerate it deliberately when you want to take upgrades. The plain `uv pip install -r requirements.txt` keeps following the unpinned requirements like pip does.
 ### Outlook / Office 365 email
 Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
 and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
 passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
 current limitation and the planned integration direction.
 ## Security Notes
 Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
 - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
 - Keep `LOCALHOST_BYPASS=false` outside local development.
 - Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
 - Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
 - Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
 - Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
 - Non-admin users do not get shell/Python/file read/write by default, and admin-only routes/tools such as MCP management, API tokens, webhooks, model/cookbook serving, backup/vault, and app settings are admin-gated. Other features are controlled by per-user privileges, so review each user's privileges before exposing a deployment.
 - Rotate any API keys or tokens that were ever pasted into a shared chat, demo, screenshot, or log.
 - If you enable API tokens or webhooks, create separate tokens per integration and delete unused ones.
 - Prefer binding manual development runs to `127.0.0.1`; bind to `0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
 - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
 - Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
 ### Private or proxied deployments
 Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
 1. Keep Odysseus on localhost, for example `127.0.0.1:7000`.
 2. Terminate HTTPS at a trusted reverse proxy or private access gateway.
 3. Put the authenticated Odysseus web/API entrypoint behind that layer.
 4. Keep raw service and model ports internal-only.
 Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
 `ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
 Common internal-only ports from the default docs/compose setup:
 | Port | Service |
 |---|---|
 | `7000` | Odysseus raw app port |
 | `8080` | SearXNG |
 | `8091` | ntfy |
 | `8100` | ChromaDB host port for manual/compose access |
 | `11434` | Ollama |
 | `8000-8020` | Common local model/provider APIs |
 ## Contributing
 Help is welcome. The best entry points are fresh-install testing, provider setup
 bugs, mobile/editor polish, docs, and small focused refactors. See
 [ROADMAP.md](ROADMAP.md) for the current help-wanted list.
-## Configuration
+Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, docs, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).
 Most setup is done inside the app with `/setup` or **Settings**. Use `.env`
 for deployment-level defaults and secrets you want present before first boot.
 Key settings:
-| Variable | Default | Description |
+## Security
 |---|---|---|
 | `LLM_HOST` | `localhost` | Your LLM server (e.g. `llm-host.local:8000`) |
 | `LLM_HOSTS` | -- | Comma-separated list for model discovery |
 | `OPENAI_API_KEY` | -- | Optional OpenAI key. Prefer adding providers in the app unless pre-seeding. |
 | `SEARXNG_INSTANCE` | `http://localhost:8080` | SearXNG URL. Docker overrides this to `http://searxng:8080`. |
 | `SEARXNG_SECRET` | generated on first Docker boot | Optional SearXNG cookie/CSRF secret. Leave blank unless you need to pin it. |
 | `APP_BIND` | `127.0.0.1` | Docker Compose host bind address for the web UI. Use `0.0.0.0` only for intentional LAN/reverse-proxy access. |
 | `APP_PORT` | `7000` | Docker Compose host port for the web UI. |
 | `APP_DATA_DIR` | `./data` | Docker Compose host directory for application data volumes. |
 | `APP_LOGS_DIR` | `./logs` | Docker Compose host directory for application logs. |
 | `AUTH_ENABLED` | `true` | Enable/disable login |
 | `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
 | `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
 | `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
 | `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
 | `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
 | `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
 | `EMBEDDING_URL` | -- | OpenAI-compatible embeddings endpoint |
 | `ODYSSEUS_CHAT_UPLOAD_MAX_BYTES` | `10485760` | Chat/agent attachment cap in bytes. Raise for larger local PDFs or text documents. |
 | `ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES` | `104857600` | Gallery image upload cap in bytes (100 MB). |
 | `ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES` | `26214400` | Gallery transform input cap in bytes (25 MB). |
 | `ODYSSEUS_MEMORY_IMPORT_MAX_BYTES` | `10485760` | Memory import file cap in bytes (10 MB). |
 | `ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES` | `26214400` | Personal document upload cap in bytes (25 MB). |
 | `ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES` | `26214400` | Email compose attachment cap in bytes (25 MB). |
 | `ODYSSEUS_STT_MAX_AUDIO_BYTES` | `26214400` | Speech-to-text audio cap in bytes (25 MB). |
 | `ODYSSEUS_ICS_MAX_BYTES` | `10485760` | Calendar `.ics` import cap in bytes (10 MB). |
-All upload-limit vars are validated (must be a positive integer) and optional; an invalid value fails fast at startup.
+Odysseus is a self-hosted workspace with powerful local tools. Keep auth enabled, keep private data out of Git, and do not expose raw model/service ports publicly. Deployment details are in the [setup guide](docs/setup.md#security-notes).
 ### Built-in MCP servers (optional setup)
 Odysseus auto-registers a few built-in MCP servers at startup. The npx-based ones (currently the browser server, `@playwright/mcp`) only start when their npm package is already in the local npx cache. If a package isn't cached, that server is skipped with a startup log message explaining what to do, so a fresh install does not block on a multi-minute npm download or hang if Playwright system deps are missing.
 To enable the browser MCP (page navigation, screenshots, vision), run once:
 ```bash
 npx -y @playwright/mcp@latest --version
 ```
 That installs `@playwright/mcp` plus Playwright (~300MB total). Restart Odysseus and the server will register at startup.
 ## Architecture
 ```
 app.py                   # FastAPI entry point
 core/      auth, database, middleware, constants
 src/       llm_core, agent_loop, agent_tools, chat_processor, search/
 routes/    chat, session, document, memory, model … endpoints
 services/  docs, memory, search, hwfit (Cookbook) …
 static/    index.html + app.js + style.css + js/ (modular front-end)
 docs/      landing page (index.html) + preview clips
 ```
 ## Data
 All user data lives in `data/` (gitignored): `app.db` (sessions, messages, documents),
 `memory.json`, `presets.json`, `uploads/`, `personal_docs/`, `chroma/`, `settings.json`.
 To back up or restore everything in `data/`, see the
 [Backup & Restore guide](docs/backup-restore.md).
 ## Star History
@@ -483,19 +72,5 @@ To back up or restore everything in `data/`, see the
 </a>
 ## License
 AGPL-3.0-or-later -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
-```
+AGPL-3.0-or-later -- see [LICENSE](LICENSE) and [ACKNOWLEDGMENTS.md](ACKNOWLEDGMENTS.md).
                                  |
                                 |||
                                |||||
                  |    |    |   |||||||
                 )_)  )_)  )_)   ~|~
                )___))___))___)\  |
               )____)____)_____)\\|
             _____|____|____|_____\\\__
             \                       /
       ~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~
               ~^~  all aboard!  ~^~
       ~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~~^~^~
 ```
@@ -331,8 +331,8 @@ if AUTH_ENABLED:
                        request.state.current_user = "internal-tool"
                    request.state.api_token = False
                    return await call_next(request)
-            except Exception:
+            except Exception as _e:
-                pass
+                logger.warning("Internal tool auth header check failed", exc_info=_e)
            # Allow DIRECT localhost requests (internal service calls from
            # heartbeats etc.). Tunnel/proxy-forwarded requests are excluded by
            # _is_trusted_loopback so LOCALHOST_BYPASS can't be abused over a
@@ -385,11 +385,10 @@ if AUTH_ENABLED:
                                    _db.close()
                            try:
                                await _asyncio.to_thread(_do)
-                            except Exception:
+                            except Exception as _e:
-                                pass
+                                logger.debug("Failed to update token last_used_at", exc_info=_e)
                        _asyncio.create_task(_touch_last_used(matched_id))
                        # Keep bearer-token callers out of normal cookie/user
                        # routes. API-aware routes can read api_token_owner.
                        request.state.current_user = "api"
                        request.state.api_token = True
                        request.state.api_token_id = matched_id
@@ -464,8 +463,8 @@ async def serve_generated_image(filename: str, request: Request):
                _db.close()
    except HTTPException:
        raise
-    except Exception:
+    except Exception as _e:
-        pass
+        logger.warning("Image ownership verification failed for %r", filename, exc_info=_e)
    ext = filename.rsplit('.', 1)[-1].lower()
    mime = {
        "png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg",
@@ -5,8 +5,9 @@ offers and pair to it, without duplicating any LLM logic.
 Auth is enforced globally by AuthMiddleware (app.py), so reaching a handler here
 means the caller is authenticated by either a cookie session or a Bearer `ody_`
-API token. The read endpoints (ping/info/models) accept either; the pairing
+API token. Ping/info accept either credential type, models requires a chat-
-endpoints are admin-cookie only.
+scoped API token for bearer callers, and the pairing endpoints are admin-cookie
 only.
 Pairing CSRF posture: minting happens ONLY on POST. The session cookie is
 SameSite=Lax (routes/auth_routes.py), which a browser does not send on a
@@ -18,7 +19,7 @@ on a GET would be unsafe (Lax cookies ride top-level GET navigations), so GET
 import html
-from fastapi import APIRouter, Request
+from fastapi import APIRouter, HTTPException, Request
 from fastapi.responses import HTMLResponse
 from core.middleware import require_admin
@@ -52,6 +53,18 @@ def owner_can_see(row_owner, owner) -> bool:
    return row_owner is None or row_owner == owner
 def require_models_scope(request: Request) -> None:
    """Require the companion chat scope for bearer-token model inventory."""
    if not getattr(request.state, "api_token", False):
        return
    scopes = getattr(request.state, "api_token_scopes", None) or []
    if isinstance(scopes, str):
        scopes = [scope.strip() for scope in scopes.split(",")]
    scope_set = {str(scope).strip() for scope in scopes if str(scope).strip()}
    if _pairing.COMPANION_SCOPE not in scope_set:
        raise HTTPException(403, "API token requires chat scope")
 def mint_pairing_token(owner: str, invalidate=None) -> tuple[str, str]:
    """Mint a pairing token AND invalidate the auth middleware's in-memory token
    cache, so the new token is accepted on the very next request without a server
@@ -103,6 +116,7 @@ def setup_companion_routes() -> APIRouter:
        rows -- the same rule as owner_filter. Read-only; never returns api_key
        material.
        """
        require_models_scope(request)
        import json as _json
        from core.database import SessionLocal, ModelEndpoint
@@ -2,12 +2,15 @@ import os
 import logging
 import sqlite3
 from datetime import datetime, timezone
 from pathlib import Path
 from sqlalchemy import event, create_engine, Column, String, Text, Boolean, DateTime, Integer, ForeignKey, JSON, Index, func, text
 from sqlalchemy.engine import Engine
 from sqlalchemy.types import TypeDecorator
 from sqlalchemy.ext.declarative import declarative_base, declared_attr
 from sqlalchemy.orm import relationship, sessionmaker, backref
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
 # Create base class for declarative models
@@ -29,9 +32,26 @@ class TimestampMixin:
    def updated_at(cls):
        return Column(DateTime, default=utcnow_naive, onupdate=utcnow_naive, nullable=False)
-# Get database URL from environment, default to SQLite in DATA_DIR
+# Ensure the writable data directory exists before SQLite connects.
 from src.constants import DATA_DIR, AUTH_FILE, MEMORY_FILE, USER_PREFS_FILE, SETTINGS_FILE
-DATABASE_URL = os.getenv("DATABASE_URL", f"sqlite:///{DATA_DIR}/app.db")
+Path(DATA_DIR).mkdir(parents=True, exist_ok=True)
 def _default_database_url() -> str:
    return f"sqlite:///{Path(DATA_DIR) / 'app.db'}"
 def _normalize_sqlite_url(url: str) -> str:
    if not url.startswith("sqlite:///"):
        return url
    db_path = url.replace("sqlite:///", "", 1)
    if db_path == ":memory:" or os.path.isabs(db_path):
        return url
    return f"sqlite:///{(Path(get_app_root()) / db_path).resolve().as_posix()}"
 # Get database URL from environment, default to SQLite in DATA_DIR
 DATABASE_URL = _normalize_sqlite_url(os.getenv("DATABASE_URL", _default_database_url()))
 # Create engine
 engine = create_engine(
@@ -324,6 +344,13 @@ class EmailAccount(TimestampMixin, Base):
    smtp_password  = Column(String, default="")
    from_address   = Column(String, default="")
    display_name   = Column(String, nullable=True)   # "Hriday Ranka" — used in From: header
    # OAuth2 (Google / Google Workspace). Tokens stored encrypted via secret_storage.
    oauth_provider      = Column(String, nullable=True)   # "google" or None
    oauth_access_token  = Column(String, nullable=True)   # encrypted
    oauth_refresh_token = Column(String, nullable=True)   # encrypted
    oauth_token_expiry  = Column(String, nullable=True)   # unix timestamp string
    __table_args__ = (
        Index('ix_email_accounts_owner_default', 'owner', 'is_default'),
@@ -1427,6 +1454,25 @@ def _migrate_add_task_automation_columns():
    except Exception as e:
        logging.getLogger(__name__).warning(f"task automation migration: {e}")
 def _migrate_add_email_oauth_columns():
    """Add Google OAuth and display_name columns to email_accounts if missing."""
    try:
        with engine.connect() as conn:
            cols = [r[1] for r in conn.execute(text("PRAGMA table_info(email_accounts)"))]
            for col, typedef in [
                ("oauth_provider",      "TEXT"),
                ("oauth_access_token",  "TEXT"),
                ("oauth_refresh_token", "TEXT"),
                ("oauth_token_expiry",  "TEXT"),
                ("display_name",        "TEXT"),
            ]:
                if col not in cols:
                    conn.execute(text(f"ALTER TABLE email_accounts ADD COLUMN {col} {typedef}"))
            conn.commit()
    except Exception as e:
        logging.getLogger(__name__).warning(f"email oauth columns migration: {e}")
 def _migrate_add_oauth_config():
    """Add oauth_config column to mcp_servers table if missing."""
    try:
@@ -1771,6 +1817,7 @@ def init_db():
    _migrate_add_tidy_verdict()
    _migrate_add_doc_source_email_cols()
    _migrate_add_oauth_config()
    _migrate_add_email_oauth_columns()
    _migrate_add_task_automation_columns()
    _migrate_add_disabled_tools()
    _migrate_add_mcp_oauth_tokens_column()
@@ -1,14 +1,16 @@
 # Security CI guide
-This project runs a set of automated security checks on every pull request and
+This project runs a set of automated security checks on pull requests and
-on every push to `main`. This page explains what each one does, whether it can
+selected branch pushes. This page explains what each one does, whether it can
 block a merge, and the few one-time settings you should turn on to get the full
 benefit.
 ## What runs, and why
-Each check lives in its own file under `.github/workflows/`. They run
+Most checks live in files under `.github/workflows/`. CodeQL is configured
-automatically; you do not start them.
+through GitHub's code scanning default setup, so it appears as a dynamic GitHub
 workflow instead of a checked-in workflow file. They run automatically; you do
 not start them.
 | Check | What it protects against | Blocks a merge? |
 |---|---|---|
@@ -88,11 +90,14 @@ let the workflows run on one pull request first, then add them here.
 2. Turn on **Dependency graph** (usually on by default for public repos) -- this
   powers Dependency review and Dependabot.
 3. Turn on **Dependabot alerts** and **Dependabot security updates**.
-4. Under **Code scanning**, you have two ways to scan the app code with CodeQL:
+4. Under **Code scanning**, use **Set up -> Default** for CodeQL. GitHub then
-   - The included `codeql.yml` workflow already scans `main` and runs weekly.
+   runs CodeQL as a dynamic workflow without the fork-token limitations that
-   - To also scan **pull requests** (recommended, since most contributions come
+   affect checked-in advanced workflows.
-     from forks), click **Set up -> Default** under Code scanning. GitHub then
+
-     runs CodeQL on pull requests for you, with no token limitations.
+   Do not also add a checked-in CodeQL workflow while default setup is enabled:
   GitHub rejects advanced CodeQL uploads when default setup is active. If the
   project later needs an advanced CodeQL workflow, disable default setup first
   and keep only one CodeQL publishing path active.
 ## Keeping it current
@@ -0,0 +1,425 @@
 # Odysseus Setup Guide
 This page keeps the detailed install, deployment, troubleshooting, and configuration notes out of the front README.
 ## Quick Start
 > **Branch note:** `dev` is the default branch and contains the latest development changes, but it may be unstable. For the more stable curated branch, use [`main`](https://github.com/pewdiepie-archdaemon/odysseus/tree/main).
 Defaults work out of the box: clone, run, then configure models/search/email
 inside **Settings**. Only edit `.env` for deployment-level overrides like
 `APP_BIND`, `APP_PORT`, `AUTH_ENABLED`, `DATABASE_URL`, or a pre-seeded admin password.
 On first setup, Odysseus creates an admin account (`admin` unless
 `ODYSSEUS_ADMIN_USER` is set) and prints a temporary password in the terminal.
 For Docker installs, the same line is in `docker compose logs odysseus`.
 Use that for the first login, then change it in **Settings**.
 Contributing? See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, testing, and
 pull request guidelines.
 ### Docker (recommended)
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 cp .env.example .env       # optional, but recommended for explicit defaults
 docker compose up -d --build
 ```
 To include optional extras in the image (PDF viewer, Office extraction; includes AGPL PyMuPDF), build with `docker compose build --build-arg INSTALL_OPTIONAL=true` before `up`.
 Open `http://localhost:7000` when the containers are healthy. Docker Compose
 binds the web UI to `127.0.0.1` by default. If the port is taken, set
 `APP_PORT=7001` in `.env` and recreate the container. Set `APP_BIND=0.0.0.0`
 only when you intentionally want LAN/reverse-proxy access.
 > **On Apple Silicon (M-series) Macs:** Docker can't reach the Metal GPU, so
 > Cookbook serves local models on CPU only. For GPU-accelerated model serving,
 > run natively instead — see [Apple Silicon](#apple-silicon) below.
 ### Native Linux / macOS
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 python3 -m venv venv
 source venv/bin/activate
 pip install -r requirements.txt
 python setup.py
 python -m uvicorn app:app --host 127.0.0.1 --port 7000
 ```
 Requirements: Python 3.11+. Cookbook also needs `tmux` for background model
 downloads and serves. The app itself is lightweight; local model serving is the
 heavy part and depends on the model, runtime, GPU, and VRAM, so small hosts can
 connect to API or remote model servers instead. Use `--host 0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
 ### Apple Silicon
 Docker on macOS cannot use the Metal GPU. For GPU-accelerated Cookbook on an
 M-series Mac, run Odysseus natively:
 ```bash
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 ./start-macos.sh
 ```
 It launches at `http://127.0.0.1:7860`. To expose it to your phone over a trusted LAN/VPN such as Tailscale, bind all interfaces:
 ```bash
 ODYSSEUS_HOST=0.0.0.0 ./start-macos.sh
 # then open http://<tailscale-ip>:7860
 ```
 The script also reads `.env` at startup, so `APP_BIND=0.0.0.0` and `APP_PORT`
 set there are picked up automatically without a command-line override each run.
 Keep `AUTH_ENABLED=true` (the default) before binding outside loopback. Do not
 expose this port directly to the public internet. To build a clickable app wrapper:
 ```bash
 ./build-macos-app.sh
 ```
 <details>
 <summary>Cookbook, GPU, Ollama, and troubleshooting notes</summary>
 **Docker bundled services.** Compose starts Odysseus, ChromaDB, SearXNG, and
 ntfy. Odysseus and the bundled service ports bind to `127.0.0.1` by default, so
 they are reachable from the host but not exposed to your LAN/public internet
 unless you opt in.
 **Cookbook storage in Docker.** Downloads live in `./data/huggingface`
 (`~/.cache/huggingface` in the container). Cookbook-installed Python CLIs and
 serve engines live in `./data/local` (`~/.local` in the container), so they
 survive container recreation.
 **Remote servers.** In **Cookbook -> Settings -> Servers**, generate the
 Odysseus SSH key and add the public key to the remote server's
 `~/.ssh/authorized_keys`. From the host you can also run:
 ```bash
 ssh-copy-id -i data/ssh/id_ed25519.pub user@server
 ```
 **Docker GPU overlays.** CPU-only users can skip this section. Cookbook can
 only detect GPUs that Docker exposes to the container — if the host runtime or
 device passthrough is not configured, Cookbook sees the iGPU, another card, or
 CPU instead of your intended GPU.
 For NVIDIA, `scripts/check-docker-gpu.sh` diagnoses GPU passthrough and can
 optionally install the host runtime or update `.env`.
 ```bash
 # Read-only diagnostic (default — installs nothing, never edits .env):
 scripts/check-docker-gpu.sh
 # Print OS-specific install commands without running them:
 scripts/check-docker-gpu.sh --print-install-commands
 # Install NVIDIA Container Toolkit on Ubuntu/Debian (requires sudo):
 scripts/check-docker-gpu.sh --install-nvidia-toolkit
 # Write COMPOSE_FILE to .env (only when GPU passthrough is confirmed working):
 scripts/check-docker-gpu.sh --enable-nvidia-overlay
 # Full assisted setup — install toolkit, then enable overlay if passthrough works:
 scripts/check-docker-gpu.sh --install-nvidia-toolkit --enable-nvidia-overlay
 ```
 Safety notes:
 - The app never installs host GPU runtime automatically.
 - The app never edits `.env` automatically.
 - `.env` is only modified when `--enable-nvidia-overlay` is explicitly passed,
  and only after GPU passthrough succeeds. `--yes` skips prompts but does not
  bypass the passthrough gate.
 - `.env.bak.*` backups created by `--enable-nvidia-overlay` are ignored by
  Git and the Docker build context.
 To enable manually without the script, add this to `.env`:
 ```bash
 COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml
 ```
 **AMD / ROCm.** AMD setup is read-only diagnostic plus manual `.env` edit. Run:
 ```bash
 scripts/check-docker-amd-gpu.sh
 ```
 Then add the reported values to `.env`, replacing `RENDER_GID` with your host's
 numeric render group id:
 ```bash
 COMPOSE_FILE=docker-compose.yml:docker/gpu.amd.yml
 RENDER_GID=989
 ```
 For NVIDIA/AMD GPU support, also read the comments in the selected overlay file: docker/gpu.nvidia.yml or docker/gpu.amd.yml.
 **Stack-management UIs (Portainer, Coolify, Dockhand, etc.).** These tools
 often accept only a single Compose file and do not reliably honor `COMPOSE_FILE`
 or multiple `-f` overlays. CLI users should keep using the `COMPOSE_FILE`
 overlay workflow above. For stack UIs, point the stack at one of the standalone
 files instead, which bundle the base stack plus the GPU settings:
 - `docker-compose.gpu-nvidia.yml` — still requires the NVIDIA Container Toolkit
  on the host.
 - `docker-compose.gpu-amd.yml` — still requires host ROCm/kfd/DRI setup, the
  `video`/`render` group membership, and `RENDER_GID` when needed.
 The base `docker-compose.yml` plus the `docker/gpu.*.yml` overlays remain the
 source of truth; the standalone files mirror them for single-file deployments.
 Verify after enabling either overlay:
 ```bash
 docker compose exec odysseus nvidia-smi -L   # NVIDIA
 docker compose exec odysseus sh -lc 'test -e /dev/kfd && test -d /dev/dri && ls -l /dev/kfd /dev/dri/renderD*'  # AMD
 ```
 > **GPU passthrough ≠ llama.cpp CUDA.** `nvidia-smi` passing inside the
 > container confirms Docker GPU access, but llama.cpp also needs `cudart` and
 > the CUDA Toolkit at runtime. If Cookbook logs show `Unable to find cudart
 > library`, `Could NOT find CUDAToolkit`, `CUDA Toolkit not found`, or
 > tensors/layers assigned to CPU, that is a Cookbook/llama.cpp build issue —
 > not a Docker passthrough failure. Reinstall the serve engine via
 > **Cookbook → Dependencies** to get a CUDA-enabled build.
 >
 > The same split applies to AMD/ROCm: seeing `/dev/kfd` and `/dev/dri` inside
 > the container confirms device passthrough, not ROCm userspace or a
 > ROCm-enabled vLLM/llama.cpp build. `rocm-smi` and `rocminfo` are not expected
 > inside the slim Odysseus image.
 **Ollama with Docker.** If Ollama runs on the host, add this endpoint in
 Settings:
 ```text
 http://host.docker.internal:11434/v1
 ```
 Ollama must listen outside its own loopback interface:
 ```bash
 OLLAMA_HOST=0.0.0.0:11434 ollama serve
 ```
 This connects Odysseus in Docker to an Ollama server that is already running on
 your host machine; it does not start Ollama inside the container.
 `host.docker.internal` is Docker's hostname for the host machine from inside the
 container. Cookbook **Serve** is a separate workflow for serving downloaded
 models through Odysseus/llama.cpp, so Windows users with an existing Ollama
 install usually only need to add the endpoint in Settings.
 **Useful checks.**
 ```bash
 docker compose ps
 docker compose logs --tail=120 odysseus
 docker compose logs odysseus | grep -E 'ChromaDB|MemoryVectorStore|DEGRADED'
 ```
 **macOS details.** `start-macos.sh` installs Homebrew deps, creates the venv,
 runs setup, and starts uvicorn on port `7860` because AirPlay often holds
 `7000`. It uses llama.cpp/Ollama for Metal. vLLM/SGLang are CUDA/ROCm-only and
 do not run on macOS. MLX-only models are not served by Odysseus.
 </details>
 ### Native Windows
 **One-command launcher** (creates the venv, installs deps, runs setup, starts the
 server; safe to re-run):
 ```powershell
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 powershell -ExecutionPolicy Bypass -File .\launch-windows.ps1
 ```
 Or do it by hand:
 ```powershell
 git clone https://github.com/pewdiepie-archdaemon/odysseus.git
 cd odysseus
 py -3.11 -m venv venv
 venv\Scripts\Activate.ps1
 pip install -r requirements.txt
 python setup.py
 python -m uvicorn app:app --host 127.0.0.1 --port 7000
 ```
 If `python` points at an older interpreter, use `py -3.12` (or another installed
 3.11+ version) for the venv step.
 **Requirements:** Python 3.11+. The core app (chat, agent, memory, documents,
 email, calendar, deep research) runs fully native. For full **Cookbook** background
 model downloads and the agent shell tool, also install
 [Git for Windows](https://git-scm.com/download/win) (provides `bash.exe`).
 Local GPU *serving* of vLLM/SGLang needs Linux/WSL2; for a local model on Windows,
 [Ollama](https://ollama.com/download) is the easiest path — point Odysseus at
 `http://localhost:11434/v1` in Settings.
 Open `http://localhost:7000`, log in with the generated admin password,
 and configure everything else inside **Settings**.
 ## Troubleshooting & Advanced Setup
 ### `chromadb-client` conflicts with embedded ChromaDB
 If `chromadb-client` (the lightweight HTTP-only package) is installed alongside the full `chromadb` package, Odysseus starts but ChromaDB silently falls back to HTTP-only mode and fails.
 **Fix:** uninstall `chromadb-client` and force-reinstall the full package:
 ```bash
 ./venv/bin/pip uninstall chromadb-client -y
 ./venv/bin/pip install --force-reinstall chromadb
 ```
 ### HTTPS + LAN/Tailscale exposure
 To expose Odysseus on a local network or Tailscale with HTTPS:
 1. Change the bind address to `0.0.0.0` in `.env` (`APP_BIND=0.0.0.0` or `ODYSSEUS_HOST=0.0.0.0`).
 2. Generate a locally-trusted cert for your LAN/Tailscale IPs using [mkcert](https://github.com/FiloSottile/mkcert):
   ```bash
   mkcert -install
   mkcert -cert-file cert.pem -key-file key.pem 192.168.1.100 tailscale-ip
   ```
 3. Run `uvicorn` with the generated certs:
   ```bash
   python -m uvicorn app:app --host 0.0.0.0 --port 7000 --ssl-certfile=cert.pem --ssl-keyfile=key.pem
   ```
 4. Install the `mkcert` CA on any other device you want to access Odysseus from (e.g., for iOS, email the `rootCA.pem` to yourself, install the profile, and trust it in Certificate Trust Settings).
 ### Optional Dependencies
 `requirements-optional.txt` contains packages that unlock extra features. It is not installed by default.
 | Package | Feature unlocked |
 |---------|-----------------|
 | `faster-whisper` | Local speech-to-text (microphone -> text) via the "local" STT provider. |
 | `ddgs` | DuckDuckGo as a search provider option. |
 | `PyMuPDF` | PDF page rendering in the side viewer panel and form-filling. (Note: AGPL-3.0) |
 | `markitdown` | Office/EPUB document text extraction (converts .docx/.xlsx/.pptx/.xls/.epub to Markdown). |
 ### Faster, reproducible installs with uv (optional)
 [uv](https://docs.astral.sh/uv/) works as a drop-in replacement for the
 venv + pip steps in the native install guides, no project changes are needed but this change results in faster installs along with a lockfile for reproducible environments. After [installing `uv`](https://docs.astral.sh/uv/getting-started/installation/), use:
 ```bash
 uv venv venv --python 3.13
 uv pip install -r requirements.txt
 # then continue as usual: python setup.py, uvicorn, ...
 ```
 `requirements.txt` is intentionally unpinned, so two installs at different times can produce different package versions. If you want a reproducible environment (e.g. across your own machines, or to roll back after a bad upgrade), snapshot and restore exact versions with:
 ```bash
 uv pip compile requirements.txt -o requirements.lock   # snapshot current resolution
 uv pip sync requirements.lock                          # reproduce it exactly later
 ```
 `requirements.lock` is gitignored and platform-specific (compile it on the OS you deploy to). Regenerate it deliberately when you want to take upgrades. The plain `uv pip install -r requirements.txt` keeps following the unpinned requirements like pip does.
 ### Outlook / Office 365 email
 Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
 and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
 passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
 current limitation and the planned integration direction.
 ## Security Notes
 Odysseus is a self-hosted workspace with powerful local tools: shell access, file uploads, model downloads, web research, email/calendar integrations, and API tokens. Treat it like an admin console.
 - Keep `AUTH_ENABLED=true` for any network-accessible deployment.
 - Keep `LOCALHOST_BYPASS=false` outside local development.
 - Use `SECURE_COOKIES=true` when Odysseus is served through HTTPS by a trusted reverse proxy or private access gateway.
 - Do not expose it directly to the public internet without HTTPS and a trusted reverse proxy or private access layer.
 - Keep `.env`, `data/`, `logs/`, databases, uploads, generated media, backups, auth/session files, API keys, and model/provider tokens out of Git and private shares. They are ignored by default.
 - Review `data/auth.json` after first boot: disable open signup unless you intentionally want it, make only your own account admin, and keep demo/test accounts non-admin.
 - Non-admin users do not get shell/Python/file read/write by default, and admin-only routes/tools such as MCP management, API tokens, webhooks, model/cookbook serving, backup/vault, and app settings are admin-gated. Other features are controlled by per-user privileges, so review each user's privileges before exposing a deployment.
 - Rotate any API keys or tokens that were ever pasted into a shared chat, demo, screenshot, or log.
 - If you enable API tokens or webhooks, create separate tokens per integration and delete unused ones.
 - Prefer binding manual development runs to `127.0.0.1`; bind to `0.0.0.0` only when you intentionally want LAN/reverse-proxy access.
 - Keep ChromaDB, SearXNG, ntfy, Ollama, vLLM, llama.cpp, databases, and raw model/provider APIs internal-only. Expose only the authenticated Odysseus web/API entrypoint through your trusted proxy or private access layer.
 - Before publishing a fork, run `git status --short` and confirm no private files from `.env`, `data/`, `logs/`, uploads, backups, or local databases are staged.
 ### Private or proxied deployments
 Odysseus serves plain HTTP on its app port. Docker Compose binds Odysseus and the bundled services to `127.0.0.1` by default, so a typical production/private setup is:
 1. Keep Odysseus on localhost, for example `127.0.0.1:7000`.
 2. Terminate HTTPS at a trusted reverse proxy or private access gateway.
 3. Put the authenticated Odysseus web/API entrypoint behind that layer.
 4. Keep raw service and model ports internal-only.
 Cloudflare Access, Tailscale, Caddy, nginx, and Traefik can all fit this pattern; none are required by Odysseus. If your access layer reaches Odysseus on the same host, proxy to `http://127.0.0.1:7000` and keep `AUTH_ENABLED=true`, `LOCALHOST_BYPASS=false`, and `SECURE_COOKIES=true`.
 `ALLOWED_ORIGINS` lists exact permitted origins for cross-origin browser/API clients; ordinary same-origin reverse-proxy access usually does not need a special CORS entry.
 Common internal-only ports from the default docs/compose setup:
 | Port | Service |
 |---|---|
 | `7000` | Odysseus raw app port |
 | `8080` | SearXNG |
 | `8091` | ntfy |
 | `8100` | ChromaDB host port for manual/compose access |
 | `11434` | Ollama |
 | `8000-8020` | Common local model/provider APIs |
 ## Configuration
 Most setup is done inside the app with `/setup` or **Settings**. Use `.env`
 for deployment-level defaults and secrets you want present before first boot.
 Key settings:
 | Variable | Default | Description |
 |---|---|---|
 | `LLM_HOST` | `localhost` | Your LLM server (e.g. `llm-host.local:8000`) |
 | `LLM_HOSTS` | -- | Comma-separated list for model discovery |
 | `OPENAI_API_KEY` | -- | Optional OpenAI key. Prefer adding providers in the app unless pre-seeding. |
 | `SEARXNG_INSTANCE` | `http://localhost:8080` | SearXNG URL. Docker overrides this to `http://searxng:8080`. |
 | `SEARXNG_SECRET` | generated on first Docker boot | Optional SearXNG cookie/CSRF secret. Leave blank unless you need to pin it. |
 | `APP_BIND` | `127.0.0.1` | Docker Compose host bind address for the web UI. Use `0.0.0.0` only for intentional LAN/reverse-proxy access. |
 | `APP_PORT` | `7000` | Docker Compose host port for the web UI. |
 | `APP_DATA_DIR` | `./data` | Docker Compose host directory for application data volumes. |
 | `APP_LOGS_DIR` | `./logs` | Docker Compose host directory for application logs. |
 | `AUTH_ENABLED` | `true` | Enable/disable login |
 | `LOCALHOST_BYPASS` | `false` | Development-only auth bypass for loopback requests. Keep false for shared/network deployments. |
 | `ALLOWED_ORIGINS` | `http://localhost,http://127.0.0.1` | Comma-separated exact permitted origins for cross-origin browser/API clients. |
 | `SECURE_COOKIES` | `false` | Set true when serving Odysseus through HTTPS at a trusted proxy or private access gateway. |
 | `DATABASE_URL` | `sqlite:///./data/app.db` | Database connection string |
 | `CHROMADB_HOST` | `localhost` | ChromaDB host for vector memory. Docker overrides this to `chromadb`. |
 | `CHROMADB_PORT` | `8100` | ChromaDB port for manual host runs. Docker overrides this to `8000`. |
 | `EMBEDDING_URL` | -- | OpenAI-compatible embeddings endpoint |
 | `ODYSSEUS_CHAT_UPLOAD_MAX_BYTES` | `10485760` | Chat/agent attachment cap in bytes. Raise for larger local PDFs or text documents. |
 | `ODYSSEUS_GALLERY_UPLOAD_MAX_BYTES` | `104857600` | Gallery image upload cap in bytes (100 MB). |
 | `ODYSSEUS_GALLERY_TRANSFORM_UPLOAD_MAX_BYTES` | `26214400` | Gallery transform input cap in bytes (25 MB). |
 | `ODYSSEUS_MEMORY_IMPORT_MAX_BYTES` | `10485760` | Memory import file cap in bytes (10 MB). |
 | `ODYSSEUS_PERSONAL_UPLOAD_MAX_BYTES` | `26214400` | Personal document upload cap in bytes (25 MB). |
 | `ODYSSEUS_EMAIL_COMPOSE_UPLOAD_MAX_BYTES` | `26214400` | Email compose attachment cap in bytes (25 MB). |
 | `ODYSSEUS_STT_MAX_AUDIO_BYTES` | `26214400` | Speech-to-text audio cap in bytes (25 MB). |
 | `ODYSSEUS_ICS_MAX_BYTES` | `10485760` | Calendar `.ics` import cap in bytes (10 MB). |
 All upload-limit vars are validated (must be a positive integer) and optional; an invalid value fails fast at startup.
 ### Built-in MCP servers (optional setup)
 Odysseus auto-registers a few built-in MCP servers at startup. The npx-based ones (currently the browser server, `@playwright/mcp`) only start when their npm package is already in the local npx cache. If a package isn't cached, that server is skipped with a startup log message explaining what to do, so a fresh install does not block on a multi-minute npm download or hang if Playwright system deps are missing.
 To enable the browser MCP (page navigation, screenshots, vision), run once:
 ```bash
 npx -y @playwright/mcp@latest --version
 ```
 That installs `@playwright/mcp` plus Playwright (~300MB total). Restart Odysseus and the server will register at startup.
 ## Architecture
 ```
 app.py                   # FastAPI entry point
 core/      auth, database, middleware, constants
 src/       llm_core, agent_loop, agent_tools, chat_processor, search/
 routes/    chat, session, document, memory, model … endpoints
 services/  docs, memory, search, hwfit (Cookbook) …
 static/    index.html + app.js + style.css + js/ (modular front-end)
 docs/      landing page (index.html) + preview clips
 ```
 ## Data
 All user data lives in `data/` (gitignored): `app.db` (sessions, messages, documents),
 `memory.json`, `presets.json`, `uploads/`, `personal_docs/`, `chroma/`, `settings.json`.
 To back up or restore everything in `data/`, see the
 [Backup & Restore guide](docs/backup-restore.md).
@@ -160,6 +160,8 @@ def setup_api_token_routes() -> APIRouter:
            payload = await request.json()
        except Exception:
            payload = {}
        if not isinstance(payload, dict):
            payload = {}
        with get_db_session() as db:
            token = db.query(ApiToken).filter(ApiToken.id == token_id).first()
            if not token:
@@ -14,7 +14,7 @@ from core.database import Session as DBSession, ModelEndpoint
 from src.llm_core import normalize_model_id
 from src.endpoint_resolver import normalize_base
 from src.context_compactor import maybe_compact, trim_for_context
-from src.auth_helpers import get_current_user
+from src.auth_helpers import effective_user
 from src.prompt_security import untrusted_context_message
 from routes.prefs_routes import _load_for_user as load_prefs_for_user
@@ -78,7 +78,7 @@ def _enforce_chat_privileges(request, sess) -> None:
    which means unrestricted allowed_models / zero cap -> no-op for them.
    """
    try:
-        user = get_current_user(request)
+        user = effective_user(request)
    except Exception:
        user = None
    if not user:
@@ -346,11 +346,11 @@ def add_user_message(sess, chat_handler, preprocessed: PreprocessedMessage, inco
 def fire_message_event(request, webhook_manager, session_id: str, sess, message: str, compare_mode: bool = False):
    """Fire webhook and event_bus events for a new user message."""
    if webhook_manager and not compare_mode:
-        asyncio.create_task(webhook_manager.fire("chat.message", {
+        webhook_manager.fire_and_forget("chat.message", {
            "session_id": session_id, "model": sess.model, "message": message[:2000],
-        }))
+        })
    from src.event_bus import fire_event
-    user = get_current_user(request)
+    user = effective_user(request)
    fire_event("message_sent", user)
@@ -577,7 +577,7 @@ async def build_chat_context(
        fire_message_event(request, webhook_manager, session_id, sess, message, compare_mode)
    # Resolve user prefs
-    user = get_current_user(request)
+    user = effective_user(request)
    uprefs = load_prefs_for_user(user)
    # Memory enabled?
@@ -1120,10 +1120,10 @@ def run_post_response_tasks(
    # Webhook
    if webhook_manager and not compare_mode:
-        asyncio.create_task(webhook_manager.fire("chat.completed", {
+        webhook_manager.fire_and_forget("chat.completed", {
            "session_id": session_id, "model": sess.model,
            "user_message": message, "response": full_response[:2000],
-        }))
+        })
    # Auto-name
    if needs_auto_name(sess.name):
@@ -23,7 +23,7 @@ from src.endpoint_resolver import normalize_base as _normalize_base, build_chat_
 from src.session_search import search_session_messages
 from src.prompt_security import untrusted_context_message
 from core.exceptions import SessionNotFoundError
-from src.auth_helpers import get_current_user
+from src.auth_helpers import effective_user, get_current_user
 from routes.session_routes import _verify_session_owner
 from routes.document_helpers import _owner_session_filter
 from core.database import SessionLocal, get_session_mode, set_session_mode
@@ -126,7 +126,8 @@ def _clear_orphaned_session_endpoint(sess, owner: str | None = None) -> bool:
        sess.model = ""
        sess.headers = {}
        return True
-    except Exception:
+    except Exception as e:
        logger.warning("Failed to clear orphaned session endpoint", exc_info=e)
        db.rollback()
        return False
    finally:
@@ -144,7 +145,8 @@ def _endpoint_cache_contains_model(endpoint, model: str) -> bool:
        return True
    try:
        models = json.loads(raw) if isinstance(raw, str) else raw
-    except Exception:
+    except Exception as e:
        logger.warning("Failed to parse cached models list, treating as containing model", exc_info=e)
        return True
    if not isinstance(models, list) or not models:
        return True
@@ -236,7 +238,8 @@ def _recover_empty_session_model(sess, session_id: str, owner: str | None = None
                is_chatgpt_subscription = False
        try:
            cached = json.loads(ep.cached_models) if isinstance(ep.cached_models, str) else (ep.cached_models or [])
-        except Exception:
+        except Exception as e:
            logger.warning("Failed to parse cached_models for endpoint %r", getattr(ep, "id", "?"), exc_info=e)
            cached = []
        if not cached:
            visible = []
@@ -360,7 +363,7 @@ def setup_chat_routes(
            sess = session_manager.get_session(session)
        except KeyError:
            raise HTTPException(404, f"Session '{session}' not found")
-        owner = get_current_user(request)
+        owner = effective_user(request)
        if _clear_orphaned_session_endpoint(sess, owner=owner):
            raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
@@ -600,7 +603,7 @@ def setup_chat_routes(
            # but BEFORE loading. Prevents cross-user session hijack.
            _verify_session_owner(request, session)
            sess = session_manager.get_session(session)
-            owner = get_current_user(request)
+            owner = effective_user(request)
            if _clear_orphaned_session_endpoint(sess, owner=owner):
                raise HTTPException(400, "Selected model endpoint was removed. Pick another model in Settings.")
            # Issue #587: picker shows a model from the endpoint cache but
@@ -631,7 +634,7 @@ def setup_chat_routes(
        _enforce_chat_privileges(request, sess)
        # Ensure session has auth headers
-        resolve_session_auth(sess, session, owner=get_current_user(request))
+        resolve_session_auth(sess, session, owner=effective_user(request))
        # Check for research_pending BEFORE mode persist overwrites it
        do_research = str(use_research).lower() == "true"
@@ -646,8 +649,8 @@ def setup_chat_routes(
        elif attachments:
            try:
                att_ids = [str(x) for x in json.loads(attachments)]
-            except Exception:
+            except Exception as e:
-                pass
+                logger.warning("Failed to parse attachments JSON, ignoring attachments", exc_info=e)
        no_memory = str(form_data.get("no_memory", "")).lower() == "true"
        pre_context_tool_policy = build_effective_tool_policy(
@@ -1482,7 +1485,7 @@ def setup_chat_routes(
        if not q or not q.strip():
            return []
-        _user = get_current_user(request)
+        _user = effective_user(request)
        return [
            result.to_dict()
            for result in search_session_messages(
@@ -790,7 +790,7 @@ def setup_codex_routes(
        norm = dict(body or {})
        sess = (norm.get("tmux_session") or norm.get("session_id") or "").strip()
        model = (norm.get("model") or norm.get("repo_id") or "").strip()
-        host = (norm.get("host") or norm.get("remote_host") or "").strip()
+        host = validate_remote_host((norm.get("host") or norm.get("remote_host") or "").strip() or None) or ""
        port = norm.get("port") or 8000
        import re as _re
        if not sess or not _re.fullmatch(r"[a-zA-Z0-9_-]+", sess):
@@ -12,6 +12,7 @@ import json
 import csv
 import io
 import os
 import inspect
 import httpx
 from pathlib import Path
 from datetime import datetime
@@ -741,8 +742,8 @@ def setup_contacts_routes():
        email = (data.get("email") or "").strip()
        phone = (data.get("phone") or "").strip()
        address = (data.get("address") or "").strip()
-        if not email and not name:
+        if not email:
-            return {"success": False, "error": "Name or email required"}
+            return {"success": False, "error": "Email required"}
        # Check if already exists by email
        if email:
            contacts = _fetch_contacts()
@@ -751,7 +752,11 @@ def setup_contacts_routes():
                    return {"success": True, "message": "Already exists", "contact": c}
        if not name:
            name = email.split("@")[0]
-        ok = _create_contact(name, email, address)
+        create_params = inspect.signature(_create_contact).parameters
        if len(create_params) >= 3:
            ok = _create_contact(name, email, address)
        else:
            ok = _create_contact(name, email)
        # If a phone was provided, do an immediate update to thread it
        # through (the simple _create_contact signature only takes name +
        # email + address; phones happen via update).
@@ -505,6 +505,8 @@ def _cached_model_scan_script(model_dirs: list[str] | None = None, add_hf_cache:
        "    if u.startswith('KB'): return int(n * 1024)",
        "    return int(n)",
        "def scan_ollama():",
        "    if any(m.get('is_ollama') for m in models): return",
        "    if os.name == 'nt' and not os.environ.get('ODYSSEUS_ALLOW_OLLAMA_CLI_SCAN'): return",
        "    if not shutil.which('ollama'): return",
        "    try:",
        "        p = subprocess.run(['ollama', 'list'], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, timeout=6)",
@@ -535,8 +537,8 @@ def _cached_model_scan_script(model_dirs: list[str] | None = None, add_hf_cache:
        "            models.append({'repo_id':name,'size_bytes':size_bytes,'nb_files':1,'has_incomplete':False,'path':'ollama','backend':'ollama','is_ollama':True})",
        "        return",
        "for _hf_cache in hf_cache_paths(): scan_hf(_hf_cache)",
        "scan_ollama()",
        "scan_ollama_api()",
        "scan_ollama()",
    ]
    for model_dir in model_dirs or []:
        lines.append(f"scan_dir(os.path.expanduser({model_dir!r}))")
@@ -503,7 +503,8 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
        user = get_current_user(request)
        try:
            data = await request.json()
-        except Exception:
+        except Exception as e:
            logger.warning("Failed to parse export request body, defaulting to empty", exc_info=e)
            data = {}
        ids = data.get("ids") or []
        if not ids:
@@ -645,8 +646,8 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
                    try:
                        from src.agent_tools.document_tools import clear_active_document
                        clear_active_document(doc_id)
-                    except Exception:
+                    except Exception as e:
-                        pass
+                        logger.warning("Failed to clear active document %r on detach", doc_id, exc_info=e)
            db.commit()
            db.refresh(doc)
            return _doc_to_dict(doc)
@@ -13,6 +13,8 @@ and `email_pollers.py` (the background loops):
 """
 import os
 import base64
 import time
 import imaplib
 import smtplib
 import email as email_mod
@@ -38,6 +40,106 @@ from src.secret_storage import decrypt as _decrypt
 logger = logging.getLogger(__name__)
 def _xoauth2_raw(user: str, access_token: str) -> str:
    """The SASL XOAUTH2 initial-response string (unencoded).
    Both smtplib.SMTP.auth() and imaplib.IMAP4.authenticate() base64-encode
    the value their callback returns, so callers pass this raw form — never
    pre-encoded — to avoid double base64.
    """
    return f"user={user}\x01auth=Bearer {access_token}\x01\x01"
 def _xoauth2_bytes(user: str, access_token: str) -> bytes:
    """Raw XOAUTH2 bytes for imaplib's authenticate() callback."""
    return _xoauth2_raw(user, access_token).encode()
 def make_oauth_state(account_id: str, owner: str) -> str:
    """Return an HMAC-signed, base64-encoded OAuth state token.
    Encodes account_id + owner + a random nonce, signed with the app secret
    so the callback can validate that the flow was initiated by an
    authenticated, owning user (CSRF / state-forgery protection).
    """
    import hmac as _hmac, hashlib as _hl, secrets as _sec
    from src.secret_storage import _load_or_create_key
    nonce = _sec.token_hex(16)
    payload = json.dumps({"a": account_id, "o": owner, "n": nonce}, separators=(",", ":"))
    sig = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
    return base64.urlsafe_b64encode(f"{payload}|{sig}".encode()).decode()
 def verify_oauth_state(state: str) -> dict | None:
    """Verify an OAuth state token's HMAC signature.
    Returns the decoded payload dict ({"a", "o", "n"}) on success, or None if
    the token is malformed, tampered, or signed with a different key.
    """
    import hmac as _hmac, hashlib as _hl
    from src.secret_storage import _load_or_create_key
    try:
        decoded = base64.urlsafe_b64decode(state.encode()).decode()
        payload, sig = decoded.rsplit("|", 1)
        expected = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
        if not _hmac.compare_digest(sig, expected):
            return None
        return json.loads(payload)
    except Exception:
        return None
 def _refresh_google_token(account_id: str) -> str | None:
    """Exchange the stored refresh token for a new access token and persist it."""
    import httpx
    from core.database import SessionLocal as _SL, EmailAccount as _EA
    from src.secret_storage import encrypt as _enc, decrypt as _dec
    client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
    client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
    if not client_id or not client_secret:
        return None
    db = _SL()
    try:
        row = db.get(_EA, account_id)
        if not row or not row.oauth_refresh_token:
            return None
        refresh_token = _dec(row.oauth_refresh_token or "")
        if not refresh_token:
            return None
        resp = httpx.post("https://oauth2.googleapis.com/token", data={
            "client_id": client_id,
            "client_secret": client_secret,
            "refresh_token": refresh_token,
            "grant_type": "refresh_token",
        }, timeout=10)
        resp.raise_for_status()
        data = resp.json()
        access_token = data["access_token"]
        row.oauth_access_token = _enc(access_token)
        row.oauth_token_expiry = str(int(time.time()) + data.get("expires_in", 3600))
        db.commit()
        return access_token
    except Exception:
        logger.warning(f"Google token refresh failed for account {account_id}")
        return None
    finally:
        db.close()
 def _get_valid_google_token(account_id: str, cfg: dict) -> str | None:
    """Return a valid Google access token, refreshing if expired or missing."""
    from src.secret_storage import decrypt as _dec
    access_token = _dec(cfg.get("oauth_access_token") or "")
    expiry_str = cfg.get("oauth_token_expiry") or ""
    if access_token and expiry_str:
        try:
            if int(expiry_str) - 60 > time.time():
                return access_token
        except (ValueError, TypeError):
            pass
    return _refresh_google_token(account_id)
 def _smtp_security_mode(cfg: dict) -> str:
    raw = str(cfg.get("smtp_security") or "").strip().lower()
    if raw in {"ssl", "starttls", "none"}:
@@ -54,20 +156,29 @@ def _send_smtp_message(cfg: dict, from_addr: str, recipients: list[str], message
    port = int(cfg.get("smtp_port") or 465)
    user = cfg.get("smtp_user") or ""
    password = cfg.get("smtp_password") or ""
    def _auth_smtp(smtp):
        if cfg.get("oauth_provider") == "google":
            token = _get_valid_google_token(cfg.get("account_id"), cfg)
            if not token:
                raise RuntimeError("Google OAuth token unavailable — reconnect the account")
            smtp.ehlo()
            smtp.auth("XOAUTH2", lambda challenge=None: _xoauth2_raw(user, token), initial_response_ok=True)
        elif user and password:
            smtp.login(user, password)
    security = _smtp_security_mode(cfg)
    if security == "ssl":
        with smtplib.SMTP_SSL(host, port, timeout=timeout) as smtp:
-            if user and password:
+            _auth_smtp(smtp)
                smtp.login(user, password)
            smtp.sendmail(from_addr, recipients, message)
        return
    with smtplib.SMTP(host, port, timeout=timeout) as smtp:
        if security == "starttls":
            smtp.starttls()
-        if user and password:
+        _auth_smtp(smtp)
            smtp.login(user, password)
        smtp.sendmail(from_addr, recipients, message)
@@ -701,10 +812,16 @@ def _get_email_config(account_id: str | None = None, owner: str = "") -> dict:
                    "imap_password": _decrypt(row.imap_password or ""),
                    "imap_starttls": bool(row.imap_starttls),
                    "from_address": row.from_address or row.imap_user or "",
                    "oauth_provider": row.oauth_provider or "",
                    "oauth_access_token": row.oauth_access_token or "",
                    "oauth_refresh_token": row.oauth_refresh_token or "",
                    "oauth_token_expiry": row.oauth_token_expiry or "",
                    "display_name": row.display_name or "",
                }
-                if not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
+                is_oauth = bool(cfg.get("oauth_provider"))
                if not is_oauth and not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
                    logger.warning(f"SMTP not configured for account {row.name!r}")
-                if not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
+                if not is_oauth and not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
                    logger.warning(f"IMAP not configured for account {row.name!r}")
                return cfg
        finally:
@@ -825,12 +942,19 @@ def _imap_connect(account_id: str | None = None, owner: str = "",
        timeout=timeout,
    )
    try:
-        conn.login(cfg["imap_user"], cfg["imap_password"])
+        if cfg.get("oauth_provider") == "google":
            token = _get_valid_google_token(cfg.get("account_id"), cfg)
            if not token:
                raise RuntimeError("Google OAuth token unavailable — reconnect the account in Settings → Integrations")
            conn.authenticate("XOAUTH2", lambda x: _xoauth2_bytes(cfg["imap_user"], token))
        else:
            conn.login(cfg["imap_user"], cfg["imap_password"])
    except Exception:
        # A failed AUTHENTICATE (e.g. an Office 365 app password on an
-        # MFA-enabled tenant, #3174) otherwise orphans the already-connected
+        # MFA-enabled tenant, #3174, or an expired/revoked OAuth token)
-        # socket; close it before propagating so a misconfigured account
+        # otherwise orphans the already-connected socket; close it before
-        # can't leak one descriptor per retry / background poller pass.
+        # propagating so a misconfigured account can't leak one descriptor
        # per retry / background poller pass.
        try:
            conn.shutdown()
        except Exception:
@@ -13,7 +13,9 @@ handlers need. The split is mechanical — no behavior change.
 """
 import asyncio
 import os
 import sqlite3 as _sql3
 import time
 import email as email_mod
 import email.header
 import email.utils
@@ -43,6 +45,7 @@ from routes.email_helpers import (
    _load_settings, _save_settings, _get_email_config,
    _send_smtp_message, _smtp_security_mode,
    _IMAP_TIMEOUT_SECONDS, _open_imap_connection,
    make_oauth_state, verify_oauth_state,
    _imap_connect, _imap, _decode_header, _detect_sent_folder, _detect_drafts_folder,
    _extract_attachment_text, _list_attachments_from_msg,
    _extract_attachment_to_disk, _extract_html, _extract_text,
@@ -76,15 +79,16 @@ def _email_tag_owner_aliases(account_id: str | None, owner: str = "") -> list[st
                        cfg.get("smtp_user") or "",
                        cfg.get("from_address") or "",
                    ])
-                except Exception:
+                except Exception as _e:
                    logger.warning("Failed to resolve email account alias", exc_info=_e)
                    resolved_account_id = None
            row = db.get(_EA, resolved_account_id) if resolved_account_id else None
            if row:
                aliases.extend([row.owner or "", row.imap_user or "", row.from_address or ""])
        finally:
            db.close()
-    except Exception:
+    except Exception as _e:
-        pass
+        logger.warning("Failed to load email aliases", exc_info=_e)
    out = []
    for a in aliases:
        a = (a or "").strip()
@@ -285,7 +289,9 @@ def _group_uid_fetch_records(msg_data) -> list:
 def _smtp_ready(cfg: dict) -> bool:
-    return bool(cfg.get("smtp_host") and cfg.get("smtp_user") and cfg.get("smtp_password"))
+    if not cfg.get("smtp_host") or not cfg.get("smtp_user"):
        return False
    return bool(cfg.get("smtp_password") or cfg.get("oauth_provider"))
 def _resolve_send_config(account_id: str | None = None, owner: str = "") -> dict:
@@ -2021,7 +2027,7 @@ def setup_email_routes():
            outer = MIMEMultipart("alternative")
            body_container = outer
-        outer["From"] = cfg["from_address"]
+        outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
        outer["To"] = to
        if cc:
            outer["Cc"] = cc
@@ -2285,6 +2291,7 @@ def setup_email_routes():
        try:
            cfg = _resolve_send_config(req.account_id, owner=owner)
        except Exception as e:
            logger.warning(f"No SMTP-capable account resolved: {e}")
            return {"success": False, "error": str(e) or "No SMTP-capable email account configured"}
        # Use 'mixed' if we have attachments, 'alternative' otherwise
@@ -2297,7 +2304,7 @@ def setup_email_routes():
            outer = MIMEMultipart("alternative")
            body_container = outer
-        outer["From"] = cfg["from_address"]
+        outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
        outer["To"] = req.to
        if req.cc:
            outer["Cc"] = req.cc
@@ -2348,6 +2355,10 @@ def setup_email_routes():
        _account_id = cfg.get("account_id") or req.account_id  # capture for the IMAP append in the closure
        _in_reply_to = (req.in_reply_to or "").strip()
        _oauth_provider = cfg.get("oauth_provider") or ""
        _oauth_access_token = cfg.get("oauth_access_token") or ""
        _oauth_refresh_token = cfg.get("oauth_refresh_token") or ""
        _oauth_token_expiry = cfg.get("oauth_token_expiry") or ""
        def _deliver():
            try:
@@ -2358,6 +2369,11 @@ def setup_email_routes():
                        "smtp_security": _smtp_security,
                        "smtp_user": _smtp_user,
                        "smtp_password": _smtp_pw,
                        "account_id": _account_id,
                        "oauth_provider": _oauth_provider,
                        "oauth_access_token": _oauth_access_token,
                        "oauth_refresh_token": _oauth_refresh_token,
                        "oauth_token_expiry": _oauth_token_expiry,
                    },
                    _from,
                    _recipients,
@@ -2470,7 +2486,7 @@ def setup_email_routes():
            msg.attach(MIMEText(_draft_html, "html", "utf-8"))
        else:
            msg = MIMEText(req.body, "plain", "utf-8")
-        msg["From"] = cfg["from_address"]
+        msg["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
        msg["To"] = req.to
        if req.cc:
            msg["Cc"] = req.cc
@@ -3122,6 +3138,8 @@ def setup_email_routes():
                    "from_address": r.from_address or "",
                    "has_imap_password": bool(r.imap_password),
                    "has_smtp_password": bool(r.smtp_password),
                    "oauth_provider": r.oauth_provider or "",
                    "display_name": r.display_name or "",
                })
            return {"accounts": out}
        finally:
@@ -3154,6 +3172,7 @@ def setup_email_routes():
                smtp_user=(data.get("smtp_user") or "").strip(),
                smtp_password=_enc(data.get("smtp_password") or ""),
                from_address=(data.get("from_address") or "").strip(),
                display_name=(data.get("display_name") or "").strip(),
                # SECURITY: stamp the creator so all subsequent reads / mutations
                # can filter by user. Without this every new account leaks to
                # every other user.
@@ -3188,7 +3207,7 @@ def setup_email_routes():
            if not row:
                return {"ok": False, "error": "Account not found"}
            # Simple fields
-            for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address"):
+            for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address", "display_name"):
                if key in data:
                    setattr(row, key, (data[key] or "").strip())
            for key in ("imap_port", "smtp_port"):
@@ -3377,4 +3396,123 @@ def setup_email_routes():
        finally:
            db.close()
    # ── Google OAuth2 routes ──
    @router.get("/oauth/google/authorize")
    async def google_oauth_authorize(account_id: str = Query(...), request: Request = None, owner: str = Depends(require_user)):
        import urllib.parse
        _assert_owns_account(account_id, owner)
        client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
        if not client_id:
            raise HTTPException(400, "GOOGLE_OAUTH_CLIENT_ID not set — add it to .env")
        redirect_uri = (
            os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
            or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
        )
        state = make_oauth_state(account_id, owner)
        params = urllib.parse.urlencode({
            "client_id": client_id,
            "redirect_uri": redirect_uri,
            "response_type": "code",
            "scope": "https://mail.google.com/ email",
            "access_type": "offline",
            "prompt": "consent",
            "state": state,
        })
        from fastapi.responses import RedirectResponse as _RR
        return _RR(f"https://accounts.google.com/o/oauth2/v2/auth?{params}")
    @router.get("/oauth/google/callback")
    async def google_oauth_callback(
        code: str = Query(None),
        state: str = Query(None),
        error: str = Query(None),
        request: Request = None,
    ):
        import urllib.parse
        from fastapi.responses import RedirectResponse as _RR
        if error:
            return _RR("/?section=integrations&email_oauth_error=google_error")
        if not code or not state:
            return _RR("/?section=integrations&email_oauth_error=missing_code")
        state_data = verify_oauth_state(state)
        if not state_data:
            return _RR("/?section=integrations&email_oauth_error=invalid_state")
        account_id = state_data.get("a", "")
        owner = state_data.get("o", "")
        client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
        client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
        redirect_uri = (
            os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
            or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
        )
        import httpx as _httpx
        try:
            resp = _httpx.post("https://oauth2.googleapis.com/token", data={
                "code": code,
                "client_id": client_id,
                "client_secret": client_secret,
                "redirect_uri": redirect_uri,
                "grant_type": "authorization_code",
            }, timeout=10)
            resp.raise_for_status()
            data = resp.json()
        except Exception:
            logger.warning("Google token exchange failed")
            return _RR("/?section=integrations&email_oauth_error=token_exchange_failed")
        access_token = data.get("access_token", "")
        refresh_token = data.get("refresh_token", "")
        expiry = str(int(time.time()) + data.get("expires_in", 3600))
        # Fetch the email address from userinfo so we can auto-fill imap_user.
        email_addr = ""
        display_name = ""
        try:
            ui = _httpx.get("https://www.googleapis.com/oauth2/v1/userinfo",
                            headers={"Authorization": f"Bearer {access_token}"}, timeout=10)
            if ui.is_success:
                ui_data = ui.json()
                email_addr = ui_data.get("email", "")
                display_name = ui_data.get("name", "")
        except Exception:
            pass
        from core.database import SessionLocal, EmailAccount
        from src.secret_storage import encrypt as _enc
        db = SessionLocal()
        try:
            row = db.query(EmailAccount).filter(EmailAccount.id == account_id).first()
            if not row:
                return _RR("/?section=integrations&email_oauth_error=account_not_found")
            # SECURITY: verify the account belongs to the initiating user.
            if owner and row.owner and row.owner != owner:
                logger.warning("OAuth callback owner mismatch — rejecting token write")
                return _RR("/?section=integrations&email_oauth_error=ownership_error")
            row.oauth_provider = "google"
            row.oauth_access_token = _enc(access_token)
            if refresh_token:
                row.oauth_refresh_token = _enc(refresh_token)
            row.oauth_token_expiry = expiry
            # Auto-fill Google IMAP/SMTP settings if not already configured.
            if not row.imap_host:
                row.imap_host = "imap.gmail.com"
                row.imap_port = 993
                row.imap_starttls = False
            if not row.smtp_host:
                row.smtp_host = "smtp.gmail.com"
                row.smtp_port = 587
            if email_addr:
                if not row.imap_user:
                    row.imap_user = email_addr
                if not row.smtp_user:
                    row.smtp_user = email_addr
                if not row.from_address:
                    row.from_address = email_addr
                if not row.name or row.name == row.id:
                    row.name = email_addr
            if display_name and not row.display_name:
                row.display_name = display_name
            db.commit()
        finally:
            db.close()
        return _RR("/?section=integrations&email_oauth_success=1")
    return router
@@ -9,6 +9,7 @@ from pathlib import Path
 from fastapi import APIRouter, HTTPException, Form, Depends
 from core.constants import EMBEDDING_ENDPOINT_FILE, FASTEMBED_CACHE_DIR
 from core.middleware import require_admin
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
@@ -67,6 +67,14 @@ def _gallery_image_path(filename: str) -> Path:
        raise HTTPException(400, "Unsafe gallery filename")
    if safe_name != original:
        raise HTTPException(400, "Unsafe gallery filename")
    if not path.exists():
        cwd_root = (Path.cwd() / "data" / "generated_images").resolve()
        cwd_path = (cwd_root / safe_name).resolve()
        try:
            if os.path.commonpath([str(cwd_root), str(cwd_path)]) == str(cwd_root) and cwd_path.exists():
                return cwd_path
        except Exception:
            pass
    return path
@@ -224,8 +232,6 @@ def setup_gallery_routes() -> APIRouter:
    @router.post("/api/gallery/{image_id}/replace")
    async def gallery_replace(request: Request, image_id: str):
        """Replace an existing gallery image file with a new one."""
        from pathlib import Path
        user = get_current_user(request)
        db = SessionLocal()
        try:
@@ -241,9 +247,8 @@ def setup_gallery_routes() -> APIRouter:
                raise HTTPException(400, "No image provided")
            content = await read_upload_limited(file, GALLERY_UPLOAD_MAX_BYTES, "Gallery replacement")
-            img_dir = Path(GENERATED_IMAGES_DIR)
+            GALLERY_IMAGE_DIR.mkdir(parents=True, exist_ok=True)
-            img_dir.mkdir(parents=True, exist_ok=True)
+            img_path = _gallery_image_path(img.filename)
            img_path = img_dir / _sanitize_gallery_filename(img.filename)
            img_path.write_bytes(content)
            # Refresh dimensions in case the editor resized the canvas.
@@ -26,7 +26,7 @@ from src.endpoint_resolver import (
    build_models_url,
    build_headers,
 )
-from src.auth_helpers import _auth_disabled, owner_filter
+from src.auth_helpers import _auth_disabled, effective_user, owner_filter
 logger = logging.getLogger(__name__)
@@ -1255,13 +1255,16 @@ def setup_model_routes(model_discovery):
        # Require auth; "" is the unconfigured single-user mode, treated as
        # "see everything" by _fetch_models.
        try:
-            from src.auth_helpers import get_current_user as _gcu
+            if getattr(request.state, "api_token", False):
-            owner = _gcu(request) or ""
+                scopes = set(getattr(request.state, "api_token_scopes", []) or [])
-        except Exception:
+                if "chat" not in scopes:
-            owner = ""
+                    raise HTTPException(403, "API token is not scoped for chat")
-        # Reject anonymous in configured deployments — no leaking the model
+                if not getattr(request.state, "api_token_owner", None):
-        # list to unauthenticated callers.
+                    raise HTTPException(403, "API token has no owner")
-        try:
+            owner = effective_user(request) or ""
            # Reject anonymous in configured deployments — no leaking the model
            # list to unauthenticated callers.
            auth_mgr = getattr(request.app.state, "auth_manager", None)
            if not owner and not _auth_disabled() and auth_mgr is not None and getattr(auth_mgr, "is_configured", False):
                raise HTTPException(401, "Not authenticated")
@@ -10,7 +10,7 @@ from fastapi import APIRouter, HTTPException, Request
 from pydantic import BaseModel
 from core.database import SessionLocal, Note
-from src.auth_helpers import get_current_user
+from src.auth_helpers import require_user
 from src.constants import DATA_DIR
 from sqlalchemy.orm.attributes import flag_modified
@@ -570,7 +570,16 @@ def setup_note_routes(task_scheduler=None):
    router = APIRouter(prefix="/api/notes", tags=["notes"])
    def _owner(request: Request) -> Optional[str]:
-        return get_current_user(request)
+        # require_user, not bare get_current_user: a request that reaches
        # these owner-scoped routes with NO identity (auth-middleware
        # regression, SSRF from a sibling service) must fail closed (401)
        # when auth is configured — not be treated as the single-user mode
        # and handed blanket access to every account's notes. The documented
        # anonymous modes (AUTH_ENABLED=false, LOCALHOST_BYPASS on loopback,
        # unconfigured first-run) still resolve to None, the single-user
        # path. fire_reminder below already gated this way; the CRUD routes
        # did not.
        return require_user(request) or None
    def _is_admin_or_single_user(request: Request, user: str | None) -> bool:
        if user == "internal-tool":
@@ -805,8 +814,7 @@ def setup_note_routes(task_scheduler=None):
        Returns {synthesis, email_sent}.
        """
        # Gate against anonymous callers — LLM synthesis can burn tokens.
-        from src.auth_helpers import require_user as _ru
+        user = require_user(request)
        user = _ru(request)
        body = await request.json()
        note_id = str(body.get("note_id") or "").strip()
        if not note_id:
@@ -278,8 +278,8 @@ def setup_personal_routes(personal_docs_manager, rag_manager, rag_available):
            # Delete file from disk if it's in uploads dir
            deleted_from_disk = False
            try:
-                abs_target = os.path.abspath(filepath)
+                abs_target = os.path.realpath(filepath)
-                base_abs = os.path.abspath(UPLOADS_DIR)
+                base_abs = os.path.realpath(UPLOADS_DIR)
                in_uploads = (
                    abs_target == base_abs
                    or os.path.commonpath([abs_target, base_abs]) == base_abs
@@ -11,7 +11,7 @@ from core.session_manager import SessionManager
 from core.models import ChatMessage
 from src.request_models import SessionResponse
 from core.database import Session as DbSession, SessionLocal, Document, GalleryImage, utcnow_naive
-from src.auth_helpers import get_current_user, effective_user, _auth_disabled, owner_filter
+from src.auth_helpers import effective_user, _auth_disabled, owner_filter
 from src.session_actions import is_session_recently_active
@@ -328,7 +328,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
        endpoint_id: str = Form(""),
    ):
        skip_val = str(skip_validation).lower() == "true"
-        user = get_current_user(request)
+        user = effective_user(request)
        endpoint_api_key = ""
        endpoint_base_url = ""
        _reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
@@ -477,7 +477,7 @@ def setup_session_routes(session_manager: SessionManager, config: dict, webhook_
                db.close()
        # Switch model/endpoint mid-session
        if model is not None and endpoint_url is not None:
-            user = get_current_user(request)
+            user = effective_user(request)
            _reject_raw_endpoint_url_for_non_admin(request, user, endpoint_id, endpoint_url)
            endpoint_api_key = ""
            endpoint_base_url = ""
@@ -7,7 +7,7 @@ from fastapi import APIRouter, Request, File, UploadFile, HTTPException
 from typing import List
 import logging
 from core.middleware import require_admin
-from src.auth_helpers import get_current_user
+from src.auth_helpers import effective_user
 from src.upload_handler import count_recent_uploads
 logger = logging.getLogger(__name__)
@@ -78,7 +78,7 @@ def setup_upload_routes(upload_handler):
        for u in files:
            try:
-                meta = upload_handler.save_upload(u, client_ip, owner=get_current_user(request))
+                meta = upload_handler.save_upload(u, client_ip, owner=effective_user(request))
                out.append({
                    "id": meta["id"],
                    "name": meta["name"],
@@ -138,7 +138,7 @@ def setup_upload_routes(upload_handler):
                original_name = info.get("name", file_id)
        auth_mgr = getattr(request.app.state, "auth_manager", None)
        auth_configured = bool(auth_mgr and auth_mgr.is_configured)
-        current_user = get_current_user(request)
+        current_user = effective_user(request)
        file_owner = info.get("owner") if info else None
        if auth_configured:
            if not current_user:
@@ -204,7 +204,7 @@ def setup_upload_routes(upload_handler):
        info = _load_upload_info(file_id)
        auth_mgr = getattr(request.app.state, "auth_manager", None)
        auth_configured = bool(auth_mgr and auth_mgr.is_configured)
-        current_user = get_current_user(request)
+        current_user = effective_user(request)
        file_owner = info.get("owner") if info else None
        if auth_configured:
            if not current_user:
@@ -247,7 +247,7 @@ def setup_upload_routes(upload_handler):
            raise HTTPException(404, "File not found")
        auth_mgr = getattr(request.app.state, "auth_manager", None)
        auth_configured = bool(auth_mgr and auth_mgr.is_configured)
-        current_user = get_current_user(request)
+        current_user = effective_user(request)
        file_owner = info.get("owner")
        if auth_configured:
            if not current_user:
@@ -1,6 +1,5 @@
 """Webhook, API Token, and sync chat routes."""
 import asyncio
 import uuid
 import logging
 from typing import Optional
@@ -385,10 +384,10 @@ def setup_webhook_routes(
        sess.add_message(ChatMessage("assistant", reply))
        session_manager.save_sessions()
-        asyncio.create_task(webhook_manager.fire("chat.completed", {
+        webhook_manager.fire_and_forget("chat.completed", {
            "session_id": session_id, "model": sess.model,
            "user_message": message[:2000], "response": reply[:2000],
-        }))
+        })
        return {"response": reply, "session_id": session_id, "model": sess.model}
@@ -19,22 +19,36 @@ GPU_BANDWIDTH = {
    "6950 xt": 576, "6900 xt": 512, "6800 xt": 512, "6800": 512, "6700 xt": 384, "6600 xt": 256, "6600": 224,
    "mi300x": 5300, "mi300": 5300, "mi250x": 3277, "mi250": 3277, "mi210": 1638, "mi100": 1229,
    "9070 xt": 624, "9070": 488, "9060 xt": 322, "9060": 322,
-    # Apple Silicon unified-memory bandwidth (GB/s). Keyed off the chip name
+    # NVIDIA GB10 Grace-Blackwell superchip (DGX Spark). Unified LPDDR5X memory,
-    # reported by sysctl machdep.cpu.brand_string (e.g. "Apple M4 Max"). Listed
+    # not Apple Silicon, so it lives in the generic GPU table — the Apple-only
-    # before the bare "m_" keys matters less than length-sorting (done below),
+    # lookup never matches it (its name carries no "apple").
-    # which guarantees "m4 max" is tried before "m4".
+    "gb10": 273,
    "m1 ultra": 800, "m1 max": 400, "m1 pro": 200, "m1": 68,
    "m2 ultra": 800, "m2 max": 400, "m2 pro": 200, "m2": 100,
    "m3 ultra": 800, "m3 max": 300, "m3 pro": 150, "m3": 100,
    "m4 max": 546, "m4 pro": 273, "m4": 120,
    "m5 max": 546, "m5 pro": 273, "m5": 150,
 }
 # Pre-sort keys by length descending for correct substring matching
 _BW_KEYS_SORTED = sorted(GPU_BANDWIDTH.keys(), key=len, reverse=True)
-# metal: backstop for Apple Silicon chips not in GPU_BANDWIDTH (e.g. a future
+# Apple Silicon unified-memory bandwidth (GB/s). For chip families with both
-# M5) — the named chips above take the accurate bandwidth path instead.
+# binned and full variants under the same "Apple Mx Max" brand string, prefer
 # GPU core count when hardware detection provides it; otherwise fall back to the
 # conservative tier so speed estimates do not over-promise.
 APPLE_BANDWIDTH_FIXED = {
    "m1 ultra": 800, "m1 max": 400, "m1 pro": 200, "m1": 68,
    "m2 ultra": 800, "m2 max": 400, "m2 pro": 200, "m2": 100,
    "m3 ultra": 800, "m3 pro": 150, "m3": 100,
    "m4 pro": 273, "m4": 120,
    "m5 pro": 307, "m5": 153,
 }
 APPLE_BANDWIDTH_BY_CORES = {
    "m3 max": {30: 300, 40: 400},
    "m4 max": {32: 410, 40: 546},
    "m5 max": {32: 460, 40: 614},
 }
 _APPLE_FIXED_KEYS_SORTED = sorted(APPLE_BANDWIDTH_FIXED.keys(), key=len, reverse=True)
 _APPLE_VARIANT_KEYS_SORTED = sorted(APPLE_BANDWIDTH_BY_CORES.keys(), key=len, reverse=True)
 # metal: backstop for Apple Silicon chips not in the explicit tables above
 # (e.g. a future M6) — use a conservative generic estimate when unknown.
 FALLBACK_K = {"cuda": 220, "rocm": 180, "metal": 150, "cpu_x86": 70, "cpu_arm": 90}
 USE_CASE_WEIGHTS = {
@@ -60,10 +74,56 @@ CONTEXT_TARGET = {
 }
-def _lookup_bandwidth(gpu_name):
+def _lookup_apple_bandwidth(system):
    gpu_name = system.get("gpu_name")
    if not isinstance(gpu_name, str) or not gpu_name:
        return None
    gn = gpu_name.lower()
    # Guard against false matches on non-Apple GPUs whose names contain
    # "m3"/"m4"/"m5" (e.g. NVIDIA Quadro M4 000).
    if "apple" not in gn:
        return None
    raw_cores = system.get("gpu_cores")
    try:
        gpu_cores = int(raw_cores) if raw_cores is not None else None
    except (TypeError, ValueError):
        gpu_cores = None
    for key in _APPLE_VARIANT_KEYS_SORTED:
        if key not in gn:
            continue
        if gpu_cores in APPLE_BANDWIDTH_BY_CORES[key]:
            return APPLE_BANDWIDTH_BY_CORES[key][gpu_cores]
        return min(APPLE_BANDWIDTH_BY_CORES[key].values())
    for key in _APPLE_FIXED_KEYS_SORTED:
        if key in gn:
            return APPLE_BANDWIDTH_FIXED[key]
    return None
 def _lookup_bandwidth(system):
    if isinstance(system, dict):
        gpu_name = system.get("gpu_name")
    else:
        gpu_name = system
    if not isinstance(gpu_name, str) or not gpu_name:
        return None
    # Apple tiers live only in the Apple-specific table now (#2564), so route
    # BOTH dict and bare-string callers through it. A bare string carries no
    # gpu_cores, so the helper falls back to the conservative (lowest) tier for
    # that model -- before #2564 the generic table answered string lookups, and
    # dropping that made _lookup_bandwidth("Apple M3 Max") return None.
    apple_input = system if isinstance(system, dict) else {"gpu_name": gpu_name}
    bw = _lookup_apple_bandwidth(apple_input)
    if bw is not None:
        return bw
    gn = gpu_name.lower()
    for key in _BW_KEYS_SORTED:
        if key in gn:
            return GPU_BANDWIDTH[key]
@@ -84,7 +144,7 @@ def _estimate_speed(model, quant, run_mode, system, offload_frac=0.0):
    """
    pb = _active_params_b(model)
    is_moe = model.get("is_moe", False)
-    bw = _lookup_bandwidth(system.get("gpu_name"))
+    bw = _lookup_bandwidth(system)
    backend = system.get("backend", "cpu_x86")
    if bw and run_mode in ("gpu", "cpu_offload"):
@@ -1,3 +1,4 @@
 import json
 import os
 import platform
 import re
@@ -335,6 +336,37 @@ def _detect_apple_silicon():
    if total_gb <= 0:
        return None
    def _parse_apple_gpu_cores(text):
        if not text:
            return None
        try:
            data = json.loads(text)
        except (TypeError, ValueError, json.JSONDecodeError):
            data = None
        if isinstance(data, dict):
            for gpu in data.get("SPDisplaysDataType") or []:
                if not isinstance(gpu, dict):
                    continue
                model = str(gpu.get("sppci_model") or gpu.get("_name") or "")
                if "apple" not in model.lower():
                    continue
                cores = gpu.get("sppci_cores")
                try:
                    return int(str(cores).strip())
                except (TypeError, ValueError):
                    continue
        m = re.search(r"Total Number of Cores:\s*(\d+)", text)
        if m:
            try:
                return int(m.group(1))
            except ValueError:
                return None
        return None
    gpu_cores = _parse_apple_gpu_cores(_run(["system_profiler", "SPDisplaysDataType", "-json"]))
    if gpu_cores is None:
        gpu_cores = _parse_apple_gpu_cores(_run(["system_profiler", "SPDisplaysDataType"]))
    # Usable GPU budget. macOS lets Metal use most of unified memory, but the
    # default working-set limit scales with RAM: small machines have to keep
    # more back for the OS + app. These fractions track Apple's
@@ -357,7 +389,7 @@ def _detect_apple_silicon():
        pass
    gpu = {"index": 0, "name": brand, "vram_gb": vram_gb}
-    return {
+    info = {
        "gpu_name": brand,
        "gpu_vram_gb": vram_gb,
        "gpu_count": 1,
@@ -369,6 +401,9 @@ def _detect_apple_silicon():
        # separate pool — downstream fit logic uses this to avoid double-budgeting.
        "unified_memory": True,
    }
    if gpu_cores is not None:
        info["gpu_cores"] = gpu_cores
    return info
 def _read_file(path):
@@ -772,6 +807,7 @@ def detect_system(host="", ssh_port="", platform="", fresh=False):
            "gpu_name": gpu_info["gpu_name"],
            "gpu_vram_gb": gpu_info["gpu_vram_gb"],
            "gpu_count": gpu_info["gpu_count"],
            "gpu_cores": gpu_info.get("gpu_cores"),
            "gpus": gpu_info.get("gpus", []),
            "gpu_groups": gpu_info.get("gpu_groups", []),
            "homogeneous": gpu_info.get("homogeneous", True),
@@ -15,6 +15,8 @@ from urllib.parse import urljoin, urlparse
 import httpx
 from bs4 import BeautifulSoup
 from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES
 from .analytics import RateLimitError, error_logger
 from .cache import (
    CONTENT_CACHE_DIR,
@@ -89,18 +91,128 @@ def _public_http_url(url: str) -> bool:
        return False
-def _get_public_url(url: str, headers: dict, timeout: int, max_redirects: int = 5) -> httpx.Response:
+class BodyTooLargeError(Exception):
    """The server declared a body larger than the hard fetch ceiling."""
    def __init__(self, url: str, declared_bytes: int):
        self.url = url
        self.declared_bytes = declared_bytes
        super().__init__(
            f"response body is {declared_bytes:,} bytes, over the "
            f"{WEB_FETCH_HARD_MAX_BYTES:,}-byte hard cap"
        )
 class _CappedFetch:
    """Result of a size-capped streaming GET.
    Carries just what fetch_webpage_content needs from an httpx.Response,
    plus the cap bookkeeping: the (possibly truncated) body, whether the
    cap cut it short, and the size the server declared via Content-Length
    (wire bytes; None when absent).
    """
    __slots__ = ("status_code", "headers", "content", "truncated",
                 "declared_bytes", "encoding", "url")
    def __init__(self, status_code, headers, content, truncated,
                 declared_bytes, encoding, url):
        self.status_code = status_code
        self.headers = headers
        self.content = content
        self.truncated = truncated
        self.declared_bytes = declared_bytes
        self.encoding = encoding
        self.url = url
    @property
    def text(self) -> str:
        return self.content.decode(self.encoding or "utf-8", errors="replace")
    def raise_for_status(self):
        if self.status_code >= 400:
            request = httpx.Request("GET", self.url)
            raise httpx.HTTPStatusError(
                f"HTTP {self.status_code} for {self.url}",
                request=request,
                response=httpx.Response(self.status_code, request=request),
            )
 def _get_public_url(url: str, headers: dict, timeout: int, max_redirects: int = 5,
                    max_bytes: int = None) -> "_CappedFetch":
    """Capped streaming GET with SSRF-guarded manual redirects.
    The body is streamed and buffering stops at ``max_bytes`` (default: the
    soft cap), so an oversized resource cannot be pulled into memory or the
    content cache in full. When Content-Length already declares a body over
    the hard ceiling, the fetch is refused before any body bytes are read.
    """
    cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
    current = url
    for _ in range(max_redirects + 1):
        if not _public_http_url(current):
            raise httpx.RequestError("Blocked private/internal URL", request=httpx.Request("GET", current))
-        response = httpx.get(current, headers=headers, timeout=timeout, follow_redirects=False)
+        # Force identity transfer-encoding. With gzip/deflate the wire bytes
-        if response.status_code not in (301, 302, 303, 307, 308):
+        # (and Content-Length) can be a small fraction of the decoded body, so
-            return response
+        # a tiny compressed response could pass the hard-cap preflight and then
-        location = response.headers.get("location")
+        # expand past the ceiling in a single decoded chunk before the streamed
-        if not location:
+        # cap below can slice it. Identity makes Content-Length the true body
-            return response
+        # size and keeps each streamed chunk bounded by the network read.
-        current = urljoin(str(response.url), location)
+        req_headers = dict(headers or {})
        req_headers["Accept-Encoding"] = "identity"
        with httpx.stream("GET", current, headers=req_headers, timeout=timeout,
                          follow_redirects=False) as response:
            if response.status_code in (301, 302, 303, 307, 308):
                location = response.headers.get("location")
                if not location:
                    return _CappedFetch(response.status_code, response.headers, b"",
                                        False, None, response.encoding, str(response.url))
                current = urljoin(str(response.url), location)
                continue
            # A server can ignore the identity request and still return a
            # compressed body; httpx.iter_bytes would then decode it, and a tiny
            # gzip can balloon into one decoded chunk far past the cap before we
            # slice. Refuse a compressed Content-Encoding so the streamed cap
            # stays a real memory bound (Content-Length is the compressed wire
            # length here, so the preflight and size metadata are unreliable too).
            enc = (response.headers.get("content-encoding") or "").strip().lower()
            if enc and enc != "identity":
                raise httpx.RequestError(
                    f"Refusing compressed response (Content-Encoding: {enc}) after "
                    "requesting identity: cannot bound decoded body size",
                    request=httpx.Request("GET", current),
                )
            declared = None
            raw_len = response.headers.get("content-length")
            if raw_len and raw_len.isdigit():
                declared = int(raw_len)
            # Refuse before buffering anything when the server already tells
            # us the body exceeds the absolute ceiling (Content-Length is wire
            # bytes; the decompressed body can only be larger).
            if declared is not None and declared > WEB_FETCH_HARD_MAX_BYTES:
                raise BodyTooLargeError(current, declared)
            chunks = []
            read = 0
            truncated = False
            # We requested identity above, so iter_bytes yields the raw body in
            # network-read-sized chunks (no decompression expansion); the cap
            # therefore bounds what we actually buffer.
            for chunk in response.iter_bytes():
                read += len(chunk)
                if read > cap:
                    keep = cap - (read - len(chunk))
                    if keep > 0:
                        chunks.append(chunk[:keep])
                    truncated = True
                    break
                chunks.append(chunk)
            return _CappedFetch(response.status_code, response.headers,
                                b"".join(chunks), truncated, declared,
                                response.encoding, str(response.url))
    raise httpx.RequestError("Too many redirects", request=httpx.Request("GET", current))
 # PDF extraction (optional dependency)
@@ -222,9 +334,19 @@ def _empty_result(url: str, error: str = "") -> dict:
 # ----------------------------------------------------------------------
 # Main content fetcher
 # ----------------------------------------------------------------------
-def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) -> dict:
+def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0,
-    """Fetch and extract meaningful content from a webpage with caching."""
+                          max_bytes: int = None) -> dict:
-    cache_key = generate_cache_key(url)
+    """Fetch and extract meaningful content from a webpage with caching.
    ``max_bytes`` raises the download budget per call (clamped to the hard
    cap); the default is the soft cap. When the body is cut short the result
    carries ``truncated``/``fetched_bytes``/``total_bytes`` so callers can
    tell the model the content is partial (#3812).
    """
    effective_cap = min(max_bytes or WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES)
    # The cap is part of the cache identity: a truncated soft-cap fetch must
    # not be served to a later full-budget request for the same URL.
    cache_key = generate_cache_key(f"{url}#cap={effective_cap}")
    cache_file = CONTENT_CACHE_DIR / f"{cache_key}.cache"
    # Check cache
@@ -250,15 +372,21 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36",
            "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
            "Accept-Language": "en-US,en;q=0.5",
-            "Accept-Encoding": "gzip, deflate",
+            # identity so the streamed size cap in _get_public_url stays honest
            # (a compressed body can decode to far more than Content-Length).
            "Accept-Encoding": "identity",
            "Connection": "keep-alive",
        }
-        response = _get_public_url(url, headers=headers, timeout=timeout)
+        response = _get_public_url(url, headers=headers, timeout=timeout,
                                   max_bytes=effective_cap)
        if response.status_code == 429:
            raise RateLimitError(f"Rate limit hit for {url} (attempt {retry_attempt})")
        response.raise_for_status()
    except BodyTooLargeError as e:
        error_logger.warning(f"Refused oversized body for {url}: {e}")
        return _empty_result(url, f"TooLarge: {e}")
    except httpx.HTTPStatusError as e:
        error_logger.warning(f"HTTP {e.response.status_code} fetching {url}: {e}")
        return _empty_result(url, f"HTTP {e.response.status_code}: {e}")
@@ -269,9 +397,27 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
        error_logger.error(str(e))
        return _empty_result(url, str(e))
    # Size bookkeeping shared by every content branch below. getattr keeps
    # plain httpx.Response stand-ins (tests) working without the cap fields.
    _size_fields = {
        "truncated": getattr(response, "truncated", False),
        "fetched_bytes": len(response.content),
        "total_bytes": getattr(response, "declared_bytes", None),
    }
    # PDF handling
    content_type = response.headers.get("Content-Type", "").lower()
    if "application/pdf" in content_type or url.lower().endswith(".pdf"):
        if _size_fields["truncated"]:
            # A PDF cut mid-stream is not parseable; unlike text there is no
            # useful partial result, so report the budget problem instead.
            _declared = _size_fields["total_bytes"]
            return _empty_result(
                url,
                f"TooLarge: PDF exceeds the {effective_cap:,}-byte fetch budget"
                + (f" (size {_declared:,} bytes)" if _declared else "")
                + "; retry with a larger budget if it fits under the hard cap",
            )
        if pdf_extract_text is None:
            logger.error("pdfminer.six is not installed; cannot extract PDF text.")
            pdf_text = ""
@@ -295,6 +441,7 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
            "js_message": "",
            "success": bool(pdf_text),
            "error": "" if pdf_text else "Failed to extract PDF text",
            **_size_fields,
        }
        _cache_result(cache_file, cache_key, result, url)
        return result
@@ -329,6 +476,7 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
            "js_message": "",
            "success": bool(text_body),
            "error": "" if text_body else "Empty response body",
            **_size_fields,
        }
        _cache_result(cache_file, cache_key, result, url)
        return result
@@ -391,6 +539,7 @@ def fetch_webpage_content(url: str, timeout: int = 5, retry_attempt: int = 0) ->
        "js_message": js_message,
        "success": True,
        "error": "",
        **_size_fields,
    }
    _cache_result(cache_file, cache_key, result, url)
    return result
@@ -9,14 +9,12 @@ from urllib.parse import urljoin, urlparse, parse_qs
 import httpx
 from bs4 import BeautifulSoup
-from src.constants import SEARXNG_INSTANCE
+from src.constants import SEARXNG_INSTANCE, REQUEST_TIMEOUT
 from .analytics import RateLimitError, error_logger
 from .query import build_enhanced_query
 logger = logging.getLogger(__name__)
 REQUEST_TIMEOUT = 20
 # Provider registry — maps setting value to (label, needs_key, needs_url)
 PROVIDER_INFO = {
    "searxng":  ("SearXNG",           False, True),
@@ -524,7 +524,7 @@ def get_builtin_overrides() -> dict:
        ov = get_setting("builtin_tool_overrides", {})
        return ov if isinstance(ov, dict) else {}
    except Exception as e:
-        logger.warning('Failed to load builtin tool overrides: %s', e)
+        logger.warning("Failed to load builtin tool overrides, using defaults", exc_info=e)
        return {}
@@ -929,8 +929,8 @@ def _build_system_prompt(
    try:
        from src.user_time import current_datetime_context_message
        _datetime_message = current_datetime_context_message()
-    except Exception:
+    except Exception as e:
-        pass
+        logger.warning("Failed to build datetime context message", exc_info=e)
    # Document context is kept as a SEPARATE message (not merged into the tool
    # prompt) so the context trimmer doesn't destroy it when truncating the
@@ -973,8 +973,8 @@ def _build_system_prompt(
            try:
                from src.pdf_form_doc import find_source_upload_id
                _is_form_backed = bool(find_source_upload_id(active_document.current_content or ""))
-            except Exception:
+            except Exception as e:
-                pass
+                logger.warning("Failed to detect if document is form-backed, assuming plain", exc_info=e)
            if _is_form_backed:
                doc_ctx = (
@@ -57,13 +57,23 @@ class WebSearchTool:
 class WebFetchTool:
    async def execute(self, content: str, ctx: dict) -> dict:
        from src.search.content import fetch_webpage_content
        from src.constants import WEB_FETCH_HARD_MAX_BYTES
        raw = content.strip()
        url = ""
        max_bytes = None
        if raw.startswith("{"):
            try:
                parsed = json.loads(raw)
                if isinstance(parsed, dict):
                    url = str(parsed.get("url") or "").strip()
                    # Download-budget override (#3812): "full": true raises the
                    # budget to the hard cap; an explicit max_bytes is clamped
                    # to the hard cap downstream. Default stays the soft cap.
                    if parsed.get("full") is True:
                        max_bytes = WEB_FETCH_HARD_MAX_BYTES
                    mb = parsed.get("max_bytes")
                    if isinstance(mb, int) and mb > 0:
                        max_bytes = mb
            except json.JSONDecodeError:
                url = ""
        if not url:
@@ -78,7 +88,7 @@ class WebFetchTool:
        loop = asyncio.get_running_loop()
        try:
            result = await asyncio.wait_for(
-                loop.run_in_executor(None, lambda: fetch_webpage_content(url, timeout=10)),
+                loop.run_in_executor(None, lambda: fetch_webpage_content(url, timeout=10, max_bytes=max_bytes)),
                timeout=30,
            )
        except asyncio.TimeoutError:
@@ -94,8 +104,28 @@ class WebFetchTool:
                return {"error": f"web_fetch: {url}: {err}", "exit_code": 1}
            return {"error": f"web_fetch: {url}: no readable text content (not HTML, or the page needs JS/login)", "exit_code": 1}
        # Tell the model when the download budget cut the body short and how
        # to get the rest, instead of silently presenting a partial page as
        # the whole thing.
        size_note = ""
        if result.get("truncated"):
            fetched = result.get("fetched_bytes") or 0
            total = result.get("total_bytes")
            total_txt = f" of {total:,} bytes" if total else ""
            size_note = (
                f"[partial content: download stopped at {fetched:,} bytes{total_txt}. "
                f'Re-call with {{"url": "{url}", "full": true}} to fetch up to '
                f"{WEB_FETCH_HARD_MAX_BYTES:,} bytes.]\n\n"
            )
        # The notice must lead the output so the MAX_OUTPUT_CHARS trim below can
        # never drop it. The title is untrusted, uncapped page content, so a
        # giant title ahead of the notice could push it out of range; keep the
        # notice first and cap the title as a second guard.
        if len(title) > 300:
            title = title[:300] + "..."
        header = (f"# {title}\n" if title else "") + f"Source: {url}\n\n"
-        output = header + text
+        output = size_note + header + text
        if len(output) > MAX_OUTPUT_CHARS:
            output = output[:MAX_OUTPUT_CHARS] + "\n\n[...truncated]"
        return {"output": output, "exit_code": 0}
@@ -1613,7 +1613,9 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
    """
    import base64
    import httpx
    import os
    from pathlib import Path
    from src.url_safety import check_outbound_url
    lines = content.strip().split("\n")
    prompt = lines[0].strip() if lines else ""
@@ -1779,8 +1781,15 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
            elif img.get("url"):
                # Download external URL and save locally (DALL-E returns temp URLs)
                result_url = img["url"]
                ok, reason = check_outbound_url(
                    result_url,
                    block_private=os.getenv("IMAGE_BLOCK_PRIVATE_IPS", "false").lower() == "true",
                )
                if not ok:
                    return {"error": f"Image API returned unsafe image URL: {reason}"}
                try:
-                    dl_resp = httpx.get(img["url"], timeout=60)
+                    dl_resp = httpx.get(result_url, timeout=60)
                    if dl_resp.status_code == 200:
                        img_dir = Path(GENERATED_IMAGES_DIR)
                        img_dir.mkdir(parents=True, exist_ok=True)
@@ -1790,10 +1799,10 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
                        image_url = f"/api/generated-image/{filename}"
                        image_id = _save_to_gallery(filename)
                    else:
-                        image_url = img["url"]  # fallback to external URL
+                        image_url = result_url  # fallback to external URL
                except Exception as _dl_e:
                    logger.warning(f"Failed to download DALL-E image: {_dl_e}")
-                    image_url = img["url"]  # fallback to external URL
+                    image_url = result_url  # fallback to external URL
            else:
                return {"error": "Image API returned unexpected format (no b64_json or url)"}
@@ -14,6 +14,7 @@ import subprocess
 import sys
 from core.platform_compat import IS_WINDOWS, which_tool
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
@@ -81,7 +82,7 @@ _BUILTIN_NPX_SERVERS = {
        "name": "Built-in: Browser",
        "command": "npx",
        "args": ["-y", "@playwright/mcp@latest", "--headless", "--caps", "vision"],
-    },
+    }
 }
 # Global flag to disable MCP if there are compatibility issues
@@ -94,7 +95,7 @@ async def register_builtin_servers(mcp_manager):
        logger.info("Built-in MCP servers disabled via ODYSSEUS_DISABLE_MCP")
        return
-    base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+    base_dir = get_app_root()
    python = sys.executable
    async def _connect_python_server(server_id: str, script_path: str, name: str):
@@ -5,6 +5,7 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
 from pydantic import Field, field_validator
 from src.constants import DATA_DIR as _DATA_DIR_CONST
 from src.runtime_paths import get_app_root
 # Cross-platform OS flag, exposed here so callers can `from src.config import
 # IS_WINDOWS`. Defined locally (a trivial `os.name == "nt"`) rather than imported
@@ -19,7 +20,7 @@ IS_WINDOWS = os.name == "nt"
 class DataConfig(BaseSettings):
    """Configuration for data storage and file handling."""
    # Base directory
-    base_dir: Path = Field(default=Path(__file__).parent.parent, description="Base directory for the application")
+    base_dir: Path = Field(default=Path(get_app_root()), description="Base directory for the application")
    # Data paths
    data_dir: Path = Field(default=Path(_DATA_DIR_CONST), description="Main data directory")
@@ -138,7 +139,7 @@ class AppConfig(BaseSettings):
        if isinstance(v, dict) and "base_dir" in v:
            base_dir = v["base_dir"]
        else:
-            base_dir = Path(__file__).parent.parent
+            base_dir = Path(get_app_root())
        # Convert string paths to Path objects relative to base_dir
        data_dir = Path(_DATA_DIR_CONST)
@@ -2,12 +2,14 @@
 """Application-wide constants and configuration values."""
 import os
 from src.runtime_paths import get_app_root, get_default_data_dir
 APP_VERSION = "1.0.0"
 # Base paths
-BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/"
+BASE_DIR = os.path.join(get_app_root(), "")
 STATIC_DIR = os.path.join(BASE_DIR, "static")
-DATA_DIR = os.getenv("ODYSSEUS_DATA_DIR", os.path.join(BASE_DIR, "data"))
+DATA_DIR = os.getenv("ODYSSEUS_DATA_DIR", get_default_data_dir())
 # Data file paths
 # Single source of truth: every persisted file/dir lives under DATA_DIR, which
@@ -63,6 +65,14 @@ MAX_OUTPUT_CHARS = 10_000       # cap for bash/python/web_search/web_fetch outpu
 MAX_READ_CHARS = 20_000         # cap for read_file / document preview
 MAX_DIFF_LINES = 400            # cap for edit_file unified-diff display
 # web_fetch response-size policy (#3812). MAX_OUTPUT_CHARS above only trims
 # what the agent SEES; these caps bound what the server downloads, parses,
 # and writes to the content cache. The soft cap is the default download
 # budget; the agent can raise it per call (full/max_bytes) but never past
 # the hard cap, so a model can't decide to pull a multi-GB file.
 WEB_FETCH_SOFT_MAX_BYTES = 2_000_000    # default download budget (2 MB)
 WEB_FETCH_HARD_MAX_BYTES = 20_000_000   # absolute ceiling, even with override (20 MB)
 # API Configuration
 MAX_CONTEXT_MESSAGES = 90
 REQUEST_TIMEOUT = 20
@@ -161,11 +161,13 @@ async def _tick() -> None:
    # Re-read state once before writing so we capture any updates from
    # concurrent UI syncs.
    stopped_any = False
    successfully_stopped_sids = set()
    for sid, host, port in to_stop:
        ok = await _stop_serve(sid, host, port)
        logger.info(f"cookbook_serve_lifecycle: stop {sid} (host={host or 'local'}): {'ok' if ok else 'failed'}")
        if ok:
            stopped_any = True
            successfully_stopped_sids.add(sid)
            # Drop the auto-registered endpoint so the model picker and
            # the chat router don't keep pointing at a dead server.
            for t in tasks:
@@ -188,12 +190,11 @@ async def _tick() -> None:
            except Exception:
                fresh = state
                fresh_tasks = tasks
            stopped_sids = {sid for sid, _, _ in to_stop}
            for ft in fresh_tasks:
                if not isinstance(ft, dict):
                    continue
                ft_sid = ft.get("sessionId") or ft.get("id")
-                if ft_sid in stopped_sids:
+                if ft_sid in successfully_stopped_sids:
                    ft["status"] = "stopped"
                    ft["_scheduledStopAtMs"] = None
                    ft["_lastStatusFlipAt"] = now_ms
@@ -31,6 +31,8 @@ import numpy as np
 import httpx
 from typing import List, Optional
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
 _DEFAULT_MODEL = "all-minilm:l6-v2"
@@ -201,11 +201,15 @@ def build_models_url(base: str) -> Optional[str]:
        return _ollama_api_root(base) + "/tags"
    if provider == "chatgpt-subscription":
        return None
-    # Generic OpenAI-compatible fallback: ensure the path lands on /v1/models
+    # Generic OpenAI-compatible fallback: local model servers with no explicit
-    # when the user omitted a path entirely. If a non-empty path is already
+    # path conventionally expose `/v1/models` (LM Studio, llama.cpp, vLLM).
-    # present (e.g. /openai, /api/openai/v1, /v1), trust the caller — the
+    # For non-local unknown hosts, do not invent `/v1`; append `/models` to the
-    # /models suffix is appended as-is and the caller's prefix is preserved.
+    # caller's base so look-alike provider hosts stay generic.
-    if not urlparse(base).path:
+    parsed = urlparse(base)
    host = (parsed.hostname or "").lower()
    is_local = host in {"localhost", "127.0.0.1", "::1", "host.docker.internal"}
    uses_v1_models_by_default = is_local or host in {"api.deepseek.com"}
    if not parsed.path and uses_v1_models_by_default:
        base = base + "/v1"
    return base + "/models"
@@ -283,7 +283,8 @@ def _is_ollama_native_url(url: str) -> bool:
    """Return True for native Ollama API URLs, including Ollama Cloud."""
    try:
        parsed = urlparse(url or "")
-    except Exception:
+    except Exception as e:
        logger.warning("Failed to parse URL for Ollama detection", exc_info=e)
        return False
    host = parsed.hostname or ""
    path = (parsed.path or "").rstrip("/")
@@ -1345,8 +1346,8 @@ def list_model_ids(
                r = httpx.get(root + "/api/tags", timeout=timeout)
                r.raise_for_status()
                return [m.get("name") or m.get("model") for m in (r.json().get("models") or []) if m.get("name") or m.get("model")]
-        except Exception:
+        except Exception as e:
-            pass
+            logger.warning("Failed to fetch model list from configured endpoint", exc_info=e)
        return []
 def normalize_model_id(
@@ -11,6 +11,8 @@ import os
 import re
 from typing import Any, Dict, List, Optional, Set, Tuple
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
 def _format_mcp_connection_error(name: str, command: str = "", args: Optional[List[str]] = None, error: Exception = None) -> str:
@@ -508,7 +510,7 @@ class McpManager:
            return False
        script_rel, name = _BUILTIN_SERVERS[server_id]
-        base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+        base_dir = get_app_root()
        script_path = os.path.join(base_dir, script_rel)
        # Clean up old connection
@@ -7,6 +7,7 @@ import time
 from pathlib import Path
 from src.constants import RAG_DIR
 from src.runtime_paths import get_app_root
 logger = logging.getLogger(__name__)
@@ -0,0 +1,30 @@
 """Helpers for resolving runtime paths in source and frozen builds."""
 import os
 import sys
 def get_app_root() -> str:
    """Return the app root directory.
    In normal source runs, this is the repository root. In a frozen Windows
    build, it is the bundle content root (PyInstaller's internal directory)
    so bundled runtime folders like `static/`, `scripts/`, and `data/` stay
    together with the executable payload.
    """
    if getattr(sys, "frozen", False):
        return getattr(sys, "_MEIPASS", os.path.dirname(os.path.abspath(sys.executable)))
    return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 def get_default_data_dir() -> str:
    """Return the default path to the data directory.
    In normal runs, this is a 'data' subdirectory under the app root.
    In frozen builds, it is a persistent user directory (~/.odysseus/data)
    to prevent SQLite databases and other persistent files from being
    written to the ephemeral, temporary extraction bundle directory.
    """
    if getattr(sys, "frozen", False):
        return os.path.join(os.path.expanduser("~"), ".odysseus", "data")
    return os.path.join(get_app_root(), "data")
@@ -1579,10 +1579,10 @@ async def do_manage_calendar(content: str, owner: Optional[str] = None) -> Dict:
        text = str(raw).strip().lower()
        if text in {"none", "no", "off", "false"}:
            return None
-        m = re.search(r"(\d+)\s*(?:m|min|minute|minutes)\b", text)
+        m = re.search(r"(\d+)\s*(?:minutes?|mins?|m)\b", text)
        if m:
            return max(0, int(m.group(1)))
-        m = re.search(r"(\d+)\s*(?:h|hr|hour|hours)\b", text)
+        m = re.search(r"(\d+)\s*(?:hours?|hrs?|h)\b", text)
        if m:
            return max(0, int(m.group(1)) * 60)
        if text.isdigit():
@@ -1595,7 +1595,7 @@ async def do_manage_calendar(content: str, owner: Optional[str] = None) -> Dict:
            return desc
        reminder_only = re.compile(
            r"^\s*(?:remind(?:er)?|alarm)\s*:?\s*\d+\s*"
-            r"(?:m|min|minute|minutes|h|hr|hour|hours)\b.*$",
+            r"(?:minutes?|mins?|m|hours?|hrs?|h)\b.*$",
            re.I,
        )
        return "" if reminder_only.match(desc) else desc
@@ -3797,7 +3797,7 @@ async def do_resolve_contact(content: str, owner: Optional[str] = None) -> Dict:
    if not name:
        return {"error": "name is required", "exit_code": 1}
-    contacts = {}  # email -> {name, source}
+    contacts = {}  # email_or_phone -> {name, source, phone?}
    # 1. CardDAV (Radicale) — structured contacts. Call in-process: a
    # server-side httpx GET to /api/contacts/search carries no session
@@ -3812,10 +3812,18 @@ async def do_resolve_contact(content: str, owner: Optional[str] = None) -> Dict:
            match = q in hay_name or any(q in (e or "").lower() for e in c.get("emails", []))
            if not match:
                continue
            has_email = False
            for email in (c.get("emails") or []):
                email = (email or "").strip().lower()
                if email and "@" in email:
                    contacts[email] = {"name": c.get("name") or email, "source": "contacts"}
                    has_email = True
            # Fall back to phone numbers when the contact has no email address
            if not has_email:
                for phone in (c.get("phones") or []):
                    phone = (phone or "").strip()
                    if phone:
                        contacts[phone] = {"name": c.get("name") or phone, "source": "contacts", "phone": phone}
    except Exception:
        pass
@@ -3835,8 +3843,11 @@ async def do_resolve_contact(content: str, owner: Optional[str] = None) -> Dict:
        return {"output": f"No contacts found matching '{name}'.", "exit_code": 0}
    lines = [f"Contacts matching '{name}':"]
-    for email, info in contacts.items():
+    for key, info in contacts.items():
-        lines.append(f"- {info['name']} <{email}> ({info['source']})")
+        if info.get("phone"):
            lines.append(f"- {info['name']} — phone: {info['phone']} ({info['source']})")
        else:
            lines.append(f"- {info['name']} <{key}> ({info['source']})")
    return {"output": "\n".join(lines), "exit_code": 0}
@@ -68,11 +68,12 @@ FUNCTION_TOOL_SCHEMAS = [
        "type": "function",
        "function": {
            "name": "web_fetch",
-            "description": "Fetch and read the text content of a specific URL the user names (e.g. 'check example.com', 'what's on this page <url>'). Use when you already have a concrete URL/domain. NOT for open-ended searches (use web_search) or 'research X' jobs (use trigger_research).",
+            "description": "Fetch and read the text content of a specific URL the user names (e.g. 'check example.com', 'what's on this page <url>'). Use when you already have a concrete URL/domain. NOT for open-ended searches (use web_search) or 'research X' jobs (use trigger_research). Downloads are size-budgeted; a '[partial content: ...]' notice in the result means the body was cut short and you can re-call with full=true for the rest.",
            "parameters": {
                "type": "object",
                "properties": {
-                    "url": {"type": "string", "description": "The URL or domain to fetch (http/https; a bare domain like example.com is fine)"}
+                    "url": {"type": "string", "description": "The URL or domain to fetch (http/https; a bare domain like example.com is fine)"},
                    "full": {"type": "boolean", "description": "Raise the download budget to the hard cap for large pages/files. Use only after a result reported partial content."}
                },
                "required": ["url"]
            }
@@ -1008,7 +1009,7 @@ FUNCTION_TOOL_SCHEMAS = [
        "type": "function",
        "function": {
            "name": "resolve_contact",
-            "description": "Look up a contact's email address by name. Searches CardDAV address book and sent email history. Use when the user says 'message [name]' or 'email [name]' without an email address.",
+            "description": "Look up a contact by name. Searches CardDAV address book and sent email history. Returns email addresses (when available) or phone numbers. Use when the user says 'message [name]', 'email [name]', or asks for someone's contact details.",
            "parameters": {
                "type": "object",
                "properties": {
@@ -1467,8 +1467,8 @@ function initEndpointForm() {
  const localAddBtn = el('adm-epLocalAddBtn');
  const localTestBtn = el('adm-epLocalTestBtn');
  if (localTestBtn) {
    const testOriginalHtml = localTestBtn.innerHTML;
    localTestBtn.addEventListener('click', async () => {
      const testOriginalHtml = localTestBtn.innerHTML || '>Test';
      const msg = _endpointMsg('local');
      msg.textContent = ''; msg.className = 'adm-ep-inline-msg';
      const raw = (el('adm-epLocalUrl').value || '').trim();
@@ -1494,8 +1494,8 @@ function initEndpointForm() {
    });
  }
  if (localAddBtn) {
    const addOriginalHtml = localAddBtn.innerHTML;
    localAddBtn.addEventListener('click', async () => {
      const addOriginalHtml = localAddBtn.innerHTML || '>Add';
      const msg = _endpointMsg('local');
      msg.textContent = ''; msg.className = 'adm-ep-inline-msg';
      const raw = (el('adm-epLocalUrl').value || '').trim();
@@ -87,7 +87,8 @@ import * as Modals from './modalManager.js';
  }
  function _accountCanSend(account) {
-    return !!(account && account.smtp_host && account.smtp_user && account.has_smtp_password);
+    if (!account || !account.smtp_host || !account.smtp_user) return false;
    return !!(account.has_smtp_password || account.oauth_provider);
  }
  async function _resolveComposeSendAccountId() {
@@ -2913,13 +2913,14 @@ async function initEmailAccountsSettings() {
    // IMAP and SMTP. Dovecot is IMAP-only here; the host is intentionally
    // blank because it may live on another machine (DNS, LAN, Tailscale).
    const PROVIDERS = {
-      gmail:    { label: 'Gmail',                  imap: { host: 'imap.gmail.com',           port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com',            port: 465 } },
+      gmail:             { label: 'Gmail',                       imap: { host: 'imap.gmail.com',        port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com',        port: 465 } },
-      migadu:   { label: 'Migadu',                 imap: { host: 'imap.migadu.com',          port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com',           port: 465 } },
+      google_workspace:  { label: 'Google Workspace / .edu',   imap: { host: 'imap.gmail.com',        port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com',        port: 587 }, oauth: 'google' },
-      icloud:   { label: 'iCloud',                 imap: { host: 'imap.mail.me.com',         port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com',          port: 587 } },
+      migadu:            { label: 'Migadu',                     imap: { host: 'imap.migadu.com',       port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com',       port: 465 } },
-      outlook:  { label: 'Outlook / Office 365',   imap: { host: 'outlook.office365.com',    port: 993, starttls: false }, smtp: { host: 'smtp.office365.com',        port: 587 } },
+      icloud:            { label: 'iCloud',                     imap: { host: 'imap.mail.me.com',      port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com',      port: 587 } },
-      fastmail: { label: 'Fastmail',               imap: { host: 'imap.fastmail.com',        port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com',         port: 465 } },
+      outlook:           { label: 'Outlook / Office 365',       imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com',    port: 587 } },
-      yahoo:    { label: 'Yahoo',                  imap: { host: 'imap.mail.yahoo.com',      port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com',       port: 465 } },
+      fastmail:          { label: 'Fastmail',                   imap: { host: 'imap.fastmail.com',     port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com',     port: 465 } },
-      dovecot:  { label: 'Dovecot IMAP (no SMTP)',  imap: { host: '',                        port: 31143, starttls: false }, smtp: { host: '',                          port: 465 } },
+      yahoo:             { label: 'Yahoo',                      imap: { host: 'imap.mail.yahoo.com',   port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com',   port: 465 } },
      dovecot:           { label: 'Dovecot IMAP (no SMTP)',     imap: { host: '',                      port: 31143, starttls: false }, smtp: { host: '',                     port: 465 } },
    };
    const _providerOptions = Object.entries(PROVIDERS)
      .map(([k, v]) => `<option value="${k}">${esc(v.label)}</option>`)
@@ -2932,11 +2933,17 @@ async function initEmailAccountsSettings() {
        <div id="eaf-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
        <div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="eaf-name" class="settings-input" placeholder="(optional — leave blank to use email)" value="${esc(a.name || '')}"></div>
        <div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="eaf-from" class="settings-input" placeholder="you@example.com" value="${esc(a.from_address || '')}"></div>
        <div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="eaf-display-name" class="settings-input" placeholder="Your Name" value="${esc(a.display_name || '')}"></div>
        <div id="eaf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
          <div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 — required for Workspace / .edu accounts</div>
          <div id="eaf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${a.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
          <button type="button" id="eaf-oauth-btn" class="admin-btn-add" style="font-size:11px">${a.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
        </div>
        <div style="font-size:11px;font-weight:600;opacity:0.6;margin:6px 0 2px">IMAP (Receiving)</div>
        <div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="eaf-imap-host" class="settings-input" value="${esc(a.imap_host || '')}"></div>
        <div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="eaf-imap-port" class="settings-input" type="number" value="${esc(a.imap_port || 993)}" style="max-width:100px"></div>
        <div class="settings-row"><label class="settings-label">Username${_hint('Usually your full email address.')}</label><input id="eaf-imap-user" class="settings-input" value="${esc(a.imap_user || '')}"></div>
-        <div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div>
+        <div class="eaf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div></div>
        <div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch"><input type="checkbox" id="eaf-imap-starttls" ${a.imap_starttls !== false ? 'checked' : ''}><span class="admin-slider"></span></label></div>
        <div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px">SMTP (Sending) <span style="font-weight:normal;opacity:0.7">— optional, leave blank for read-only</span></div>
        <div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com, smtp.migadu.com. Leave blank to make this account read-only.')}</label><input id="eaf-smtp-host" class="settings-input" value="${esc(a.smtp_host || '')}"></div>
@@ -2959,6 +2966,16 @@ async function initEmailAccountsSettings() {
      </div>
    `;
    // Show/hide OAuth section and password fields based on provider selection.
    function _syncOauthUI(providerKey) {
      const p = PROVIDERS[providerKey];
      const isOauth = !!(p && p.oauth);
      el('eaf-oauth-section').style.display = isOauth ? '' : 'none';
      formEl.querySelectorAll('.eaf-password-section').forEach(r => {
        r.style.display = isOauth ? 'none' : '';
      });
    }
    const eafProviderNotes = {
      outlook: {
        title: 'Outlook / Office 365 needs OAuth',
@@ -2983,13 +3000,41 @@ async function initEmailAccountsSettings() {
    el('eaf-provider').addEventListener('change', (e) => {
      _renderEafProviderNote(e.target.value);
      const p = PROVIDERS[e.target.value];
-      if (!p) return;
+      if (!p) { _syncOauthUI(''); return; }
      el('eaf-imap-host').value = p.imap.host;
      el('eaf-imap-port').value = p.imap.port;
      el('eaf-imap-starttls').checked = !!p.imap.starttls;
      el('eaf-smtp-host').value = p.smtp.host;
      el('eaf-smtp-port').value = p.smtp.port;
      el('eaf-smtp-security').value = p.smtp.security || ((parseInt(p.smtp.port || 465) === 587) ? 'starttls' : 'ssl');
      _syncOauthUI(e.target.value);
    });
    // Init OAuth UI for accounts already connected via OAuth.
    if (a.oauth_provider === 'google') _syncOauthUI('google_workspace');
    // "Connect with Google" button — save the account first, then redirect to OAuth.
    el('eaf-oauth-btn').addEventListener('click', async () => {
      // Must save the account first to get an account_id to pass to the OAuth flow.
      const body = {
        name: el('eaf-name').value.trim() || el('eaf-from').value.trim(),
        from_address: el('eaf-from').value.trim(),
        imap_host: el('eaf-imap-host').value.trim(),
        imap_port: parseInt(el('eaf-imap-port').value) || 993,
        imap_user: el('eaf-imap-user').value.trim(),
        imap_starttls: el('eaf-imap-starttls').checked,
        smtp_host: el('eaf-smtp-host').value.trim(),
        smtp_port: parseInt(el('eaf-smtp-port').value) || 587,
        smtp_user: el('eaf-imap-user').value.trim(),
      };
      if (!body.name) { el('eaf-msg').textContent = 'Enter a Name or Email first'; el('eaf-msg').style.color = 'var(--red)'; return; }
      const url = isEdit ? `/api/email/accounts/${a.id}` : '/api/email/accounts';
      const method = isEdit ? 'PUT' : 'POST';
      const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
      const d = await r.json();
      if (!d.ok) { el('eaf-msg').textContent = d.error || 'Save failed'; el('eaf-msg').style.color = 'var(--red)'; return; }
      const accId = isEdit ? a.id : d.id;
      window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
    });
    el('eaf-smtp-security').value = _smtpSecurity(a);
@@ -3009,6 +3054,7 @@ async function initEmailAccountsSettings() {
      const body = {
        name: el('eaf-name').value.trim(),
        from_address: el('eaf-from').value.trim(),
        display_name: el('eaf-display-name').value.trim(),
        imap_host: el('eaf-imap-host').value.trim(),
        imap_port: parseInt(el('eaf-imap-port').value) || 993,
        imap_user: el('eaf-imap-user').value.trim(),
@@ -4317,6 +4363,7 @@ async function initUnifiedIntegrations() {
    // it may be remote (DNS, LAN, Tailscale), not localhost.
    const PROVIDERS = {
      gmail:    { label: 'Gmail',                   emailEx: 'you@gmail.com',     imap: { host: 'imap.gmail.com',           port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com',     port: 465 } },
      google_workspace: { label: 'Google Workspace / .edu', emailEx: 'you@yourschool.edu', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 587 }, oauth: 'google' },
      migadu:   { label: 'Migadu',                  emailEx: 'you@yourdomain.com', imap: { host: 'imap.migadu.com',          port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com',    port: 465 } },
      icloud:   { label: 'iCloud',                  emailEx: 'you@icloud.com',    imap: { host: 'imap.mail.me.com',         port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com',   port: 587 } },
      outlook:  { label: 'Outlook / Office 365',    emailEx: 'you@outlook.com',   imap: { host: 'outlook.office365.com',    port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
@@ -4334,6 +4381,7 @@ async function initUnifiedIntegrations() {
    const PROV_LOGO = {
      '':       _customLogo,
      gmail:    _letterLogo('G', '#ea4335'),
      google_workspace: _letterLogo('G', '#ea4335'),
      migadu:   _letterLogo('M', '#3aa39d'),
      icloud:   _letterLogo('i', '#3693f3'),
      outlook:  _letterLogo('O', '#0078d4'),
@@ -4362,11 +4410,17 @@ async function initUnifiedIntegrations() {
          <div id="uf-email-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
          <div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="uf-email-name" class="settings-input" placeholder="(optional — leave blank to use email)"></div>
          <div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="uf-email-from" class="settings-input" placeholder="you@example.com"></div>
          <div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="uf-display-name" class="settings-input" placeholder="Your Name"></div>
          <div id="uf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
            <div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 — required for Workspace / .edu accounts</div>
            <div id="uf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${existing && existing.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
            <button type="button" id="uf-oauth-btn" class="admin-btn-add" style="font-size:11px">${existing && existing.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
          </div>
          <div style="font-size:11px;font-weight:600;opacity:0.6;margin:4px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><polyline points="22 12 16 12 14 15 10 15 8 12 2 12"/><path d="M5.45 5.11 2 12v6a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-6l-3.45-6.89A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z"/></svg>IMAP (Receiving)</div>
          <div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="uf-imap-host" class="settings-input" placeholder="imap.example.com"></div>
          <div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="uf-imap-port" class="settings-input" type="number" placeholder="993" style="max-width:100px"></div>
          <div class="settings-row"><label class="settings-label">Username${_hint('Yes — your full email address goes here too (e.g. you@gmail.com). Same as the Email field above for almost every provider.')}</label><input id="uf-imap-user" class="settings-input" placeholder="you@example.com"></div>
-          <div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div>
+          <div class="uf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div></div>
          <div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch" style="margin-left:0"><input type="checkbox" id="uf-imap-starttls" checked><span class="admin-slider"></span></label></div>
          <div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><line x1="22" y1="2" x2="11" y2="13"/><polygon points="22 2 15 22 11 13 2 9 22 2"/></svg>SMTP (Sending) <span style="font-weight:normal;opacity:0.7">— optional, leave blank for read-only</span></div>
          <div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com. Leave blank to make this account read-only.')}</label><input id="uf-smtp-host" class="settings-input" placeholder="smtp.example.com"></div>
@@ -4491,6 +4545,16 @@ async function initUnifiedIntegrations() {
        </div>`;
    };
    // Show/hide the OAuth section and password fields based on provider selection.
    function _syncOauthUI(providerKey) {
      const p = PROVIDERS[providerKey];
      const isOauth = !!(p && p.oauth);
      el('uf-oauth-section').style.display = isOauth ? '' : 'none';
      formEl.querySelectorAll('.uf-password-section').forEach(r => {
        r.style.display = isOauth ? 'none' : '';
      });
    }
    // Custom dropdown wire-up — the native <select> stays in the DOM as the
    // data source and accessibility target, but the visible UI is a button +
    // popup so each provider row can render with its SVG logo. Selecting an
@@ -4547,6 +4611,7 @@ async function initUnifiedIntegrations() {
    el('uf-email-provider').addEventListener('change', (e) => {
      const key = e.target.value;
      _renderProviderNote(key);
      _syncOauthUI(key);
      const p = PROVIDERS[key];
      if (!p) return;
      el('uf-imap-host').value = p.imap.host;
@@ -4562,6 +4627,23 @@ async function initUnifiedIntegrations() {
      }
    });
    // Init OAuth UI for accounts already connected via OAuth.
    if (existing && existing.oauth_provider === 'google') _syncOauthUI('google_workspace');
    // "Connect with Google" — save the account first, then redirect to OAuth.
    el('uf-oauth-btn').addEventListener('click', async () => {
      const body = _collectBody();
      if (!body.name) body.name = body.from_address;
      if (!body.name) { el('uf-email-msg').textContent = 'Enter a Name or Email first'; el('uf-email-msg').style.color = 'var(--red)'; return; }
      const url = isEdit ? `/api/email/accounts/${editId}` : '/api/email/accounts';
      const method = isEdit ? 'PUT' : 'POST';
      const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
      const d = await r.json();
      if (!(d.ok || d.id)) { el('uf-email-msg').textContent = d.error || 'Save failed'; el('uf-email-msg').style.color = 'var(--red)'; return; }
      const accId = isEdit ? editId : d.id;
      window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
    });
    // "Same as IMAP" toggle — hide the SMTP creds rows when on.
    const _syncSmtpSame = () => {
      const same = el('uf-smtp-same').checked;
@@ -4574,6 +4656,7 @@ async function initUnifiedIntegrations() {
    if (existing) {
      el('uf-email-name').value = existing.name || '';
      el('uf-email-from').value = existing.from_address || '';
      el('uf-display-name').value = existing.display_name || '';
      el('uf-imap-host').value = existing.imap_host || '';
      el('uf-imap-port').value = existing.imap_port || 993;
      el('uf-imap-user').value = existing.imap_user || '';
@@ -4622,6 +4705,7 @@ async function initUnifiedIntegrations() {
      const body = {
        name: el('uf-email-name').value.trim(),
        from_address: el('uf-email-from').value.trim(),
        display_name: el('uf-display-name').value.trim(),
        imap_host: el('uf-imap-host').value.trim(),
        imap_port: parseInt(el('uf-imap-port').value) || 993,
        imap_user: el('uf-imap-user').value.trim(),
@@ -5650,6 +5734,40 @@ export function close() {
  }
 }
 // Handle redirect back from Google OAuth2 — open settings to integrations and show status.
 (function _handleOauthRedirect() {
  const sp = new URLSearchParams(window.location.search);
  if (!sp.has('email_oauth_success') && !sp.has('email_oauth_error')) return;
  // Strip params from URL without a page reload.
  const clean = window.location.pathname + window.location.hash;
  window.history.replaceState(null, '', clean);
  const success = sp.has('email_oauth_success');
  const errMsg = sp.get('email_oauth_error') || '';
  // Open settings → integrations after the app has initialised.
  function _tryOpen() {
    if (window.settingsModule && typeof window.settingsModule.open === 'function') {
      window.settingsModule.open('integrations');
      // Brief toast-style banner.
      const banner = document.createElement('div');
      banner.textContent = success
        ? '✓ Google account connected — email is ready'
        : `Google OAuth failed: ${errMsg || 'unknown error'}`;
      Object.assign(banner.style, {
        position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
        background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
        color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
        fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
        boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
      });
      document.body.appendChild(banner);
      setTimeout(() => banner.remove(), 4000);
    } else {
      setTimeout(_tryOpen, 100);
    }
  }
  _tryOpen();
 })();
 const settingsModule = { open, close, initIntegrations, initUnifiedIntegrations, syncAdminVisibility, refreshAiModelEndpoints };
@@ -0,0 +1,326 @@
 # Oversized Test File Split Plan
 ## Purpose
 This document plans future oversized test-file splits using current repo data.
 It does not move files, rewrite assertions, extract helpers, or change CI.
 ## Roadmap context
 - Issue: #3983
 - Parent tracker: #2523
 - Follows #3973 / #3982, the report-only order-sensitivity diagnostics slice.
 ## Methodology
 Metrics were generated from the current test tree using:
 - physical line counts for every recursive `test_*.py` file under `tests/`;
 - AST counts for `test_*` functions and `Test*` classes;
 - one `pytest --collect-only -q tests` run to count collected items per file;
 - current taxonomy classification from `tests._taxonomy.classify_test_path`; and
 - static setup-signal scans for route/API, DB/session, import-state, security, filesystem, subprocess/script, async/threading, and UI/static indicators.
 Static signals are not proof of risk. They are review prompts.
 Future split PRs must still inspect each file manually before editing.
 ## Current summary
 - test files scanned: 583
 - collected pytest items counted: 3586
 - large-file threshold: 300 lines
 - large-collected threshold: 20 collected items
 Area distribution:
 | Value | Files |
 |---|---:|
 | cli | 28 |
 | helpers | 1 |
 | js | 39 |
 | routes | 23 |
 | security | 77 |
 | services | 144 |
 | uncategorized | 234 |
 | unit | 37 |
 Sub-area distribution:
 | Value | Files |
 |---|---:|
 | api | 6 |
 | atomic | 3 |
 | auth | 9 |
 | calendar | 10 |
 | cli | 28 |
 | confinement | 7 |
 | cookbook | 13 |
 | document | 11 |
 | email | 12 |
 | embedding | 3 |
 | gallery | 5 |
 | history | 3 |
 | js | 39 |
 | llm | 16 |
 | mcp | 8 |
 | memory | 15 |
 | nondict | 7 |
 | nonstring | 22 |
 | owner | 14 |
 | owner_scope | 23 |
 | parse | 4 |
 | provider | 6 |
 | research | 16 |
 | route | 6 |
 | routes | 9 |
 | scheduler | 3 |
 | scope | 5 |
 | security | 9 |
 | session | 16 |
 | ssrf | 3 |
 | webhook | 3 |
 | xss | 5 |
 Values below 2 files: 244 values covering 244 files.
 ## Top files by collected pytest items
 | File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
 |---|---:|---:|---:|---:|---|---|---|
 | `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
 | `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
 | `tests/test_provider_classification.py` | 188 | 67 | 21 | 4 | services | provider | - |
 | `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
 | `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
 | `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
 | `tests/test_provider_endpoints.py` | 241 | 58 | 18 | 1 | services | provider | subprocess/script |
 | `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
 | `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
 | `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
 | `tests/test_llm_core_temperature.py` | 196 | 41 | 17 | 0 | services | llm | - |
 | `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
 | `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | 6 | 0 | services | llm | db/session |
 | `tests/test_chat_helpers.py` | 264 | 31 | 18 | 0 | uncategorized | chat_helpers | route/api |
 | `tests/test_provider_detection.py` | 148 | 31 | 31 | 5 | services | provider | - |
 | `tests/test_model_context.py` | 251 | 30 | 30 | 4 | uncategorized | model_context | db/session, import-state |
 | `tests/test_endpoint_resolver.py` | 148 | 30 | 30 | 6 | uncategorized | endpoint_resolver | - |
 | `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
 | `tests/test_upload_limits_centralized.py` | 110 | 29 | 5 | 0 | uncategorized | upload_limits_centralized | import-state, filesystem |
 | `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
 | `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
 | `tests/test_taxonomy.py` | 145 | 26 | 16 | 0 | uncategorized | taxonomy | security, ui/static |
 | `tests/test_tool_path_confinement.py` | 282 | 24 | 24 | 0 | security | confinement | import-state, filesystem, async/threading |
 | `tests/test_copilot.py` | 170 | 23 | 16 | 0 | uncategorized | copilot | - |
 | `tests/test_research_utils.py` | 97 | 23 | 23 | 2 | services | research | - |
 | `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_tool_support_heuristic.py` | 166 | 22 | 22 | 3 | uncategorized | tool_support_heuristic | - |
 | `tests/test_platform_compat.py` | 318 | 21 | 21 | 0 | uncategorized | platform_compat | import-state, filesystem, subprocess/script |
 ## Top files by physical line count
 | File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |
 |---|---:|---:|---:|---:|---|---|---|
 | `tests/test_model_routes.py` | 1778 | 139 | 116 | 10 | routes | routes | route/api, db/session, import-state, async/threading |
 | `tests/test_security_regressions.py` | 1224 | 92 | 68 | 0 | security | security | route/api, db/session, import-state, security, filesystem, async/threading, ui/static |
 | `tests/test_embedding_lanes.py` | 1104 | 29 | 29 | 0 | services | embedding | filesystem |
 | `tests/test_pr_blocker_audit.py` | 964 | 58 | 58 | 0 | uncategorized | pr_blocker_audit | import-state, security, filesystem |
 | `tests/test_review_regressions.py` | 930 | 26 | 26 | 0 | uncategorized | review_regressions | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_cookbook_helpers.py` | 912 | 65 | 65 | 0 | services | cookbook | route/api, filesystem, subprocess/script, async/threading, ui/static |
 | `tests/test_rename_user_owner_sync.py` | 686 | 26 | 26 | 0 | security | owner | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_email_oauth.py` | 580 | 28 | 25 | 0 | services | email | route/api, db/session, security, async/threading |
 | `tests/test_api_token_routes.py` | 578 | 17 | 17 | 0 | routes | api_routes | route/api, db/session, import-state, async/threading |
 | `tests/test_shell_routes.py` | 481 | 63 | 48 | 8 | routes | routes | route/api, import-state, filesystem |
 | `tests/test_email_owner_scope.py` | 474 | 9 | 9 | 0 | security | owner_scope | route/api, db/session, filesystem, async/threading |
 | `tests/test_service_health.py` | 472 | 47 | 42 | 0 | uncategorized | service_health | async/threading |
 | `tests/test_agent_loop.py` | 469 | 52 | 52 | 5 | uncategorized | agent_loop | db/session, import-state |
 | `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | 8 | 0 | uncategorized | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading |
 | `tests/test_helpers_import_state.py` | 426 | 26 | 26 | 0 | helpers | helpers | route/api, db/session, import-state |
 | `tests/test_endpoint_owner_scope_followup.py` | 414 | 11 | 11 | 0 | security | owner_scope | route/api, db/session, filesystem |
 | `tests/test_endpoint_probing.py` | 411 | 34 | 30 | 6 | uncategorized | endpoint_probing | route/api, db/session, import-state |
 | `tests/test_imap_leak_fixes.py` | 404 | 15 | 15 | 0 | uncategorized | imap_leak_fixes | route/api, db/session, security, filesystem |
 | `tests/test_companion_readonly.py` | 402 | 17 | 17 | 0 | uncategorized | companion_readonly | db/session, import-state |
 | `tests/test_api_chat_security.py` | 401 | 22 | 8 | 0 | security | security | route/api, db/session, import-state, filesystem, async/threading |
 | `tests/test_upload_handler_atomicity.py` | 401 | 9 | 9 | 0 | uncategorized | upload_handler_atomicity | filesystem, async/threading |
 | `tests/test_run_focus.py` | 399 | 47 | 44 | 0 | uncategorized | run_focus | security, filesystem, subprocess/script, ui/static |
 | `tests/test_auth_regressions.py` | 375 | 15 | 15 | 0 | security | auth | route/api, db/session, import-state, async/threading |
 | `tests/test_calendar_owner_scope.py` | 345 | 7 | 7 | 0 | security | owner_scope | route/api, db/session, import-state, filesystem, async/threading, ui/static |
 | `tests/test_null_owner_gates.py` | 342 | 20 | 20 | 0 | security | owner | route/api, db/session, import-state |
 | `tests/test_agent_migration_manifest.py` | 340 | 15 | 15 | 0 | uncategorized | agent_migration_manifest | import-state, filesystem |
 | `tests/test_calendar_recurrence.py` | 338 | 19 | 19 | 0 | services | calendar | - |
 | `tests/test_tool_policy.py` | 330 | 13 | 13 | 0 | uncategorized | tool_policy | import-state, async/threading |
 | `tests/test_workspace_confine.py` | 328 | 18 | 18 | 0 | uncategorized | workspace_confine | route/api, filesystem, subprocess/script, async/threading |
 | `tests/test_diffusion_server_security.py` | 325 | 14 | 14 | 0 | security | security | route/api, import-state, security, filesystem, async/threading, ui/static |
 ## Split planning candidates
 This section is generated from metrics, not from manual judgement.
 Files are included when they meet at least one threshold:
 - at least 300 physical lines; or
 - at least 20 collected pytest items.
 These are planning candidates only. A later split PR still needs a focused manual review of each file before moving tests.
 | File | Why included | Setup/risk signals | Suggested handling |
 |---|---|---|---|
 | `tests/test_model_routes.py` | 1778 lines, 139 collected tests | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_security_regressions.py` | 1224 lines, 92 collected tests | route/api, db/session, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_provider_classification.py` | 67 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_cookbook_helpers.py` | 912 lines, 65 collected tests | route/api, filesystem, subprocess/script, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_shell_routes.py` | 481 lines, 63 collected tests | route/api, import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_pr_blocker_audit.py` | 964 lines, 58 collected tests | import-state, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_provider_endpoints.py` | 58 collected tests | subprocess/script | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_agent_loop.py` | 469 lines, 52 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_service_health.py` | 472 lines, 47 collected tests | async/threading | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_run_focus.py` | 399 lines, 47 collected tests | security, filesystem, subprocess/script, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_llm_core_temperature.py` | 41 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_endpoint_probing.py` | 411 lines, 34 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_llm_core_anthropic_temp_omit.py` | 32 collected tests | db/session | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_chat_helpers.py` | 31 collected tests | route/api | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_provider_detection.py` | 31 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_model_context.py` | 30 collected tests | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_endpoint_resolver.py` | 30 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_embedding_lanes.py` | 1104 lines, 29 collected tests | filesystem | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_upload_limits_centralized.py` | 29 collected tests | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_email_oauth.py` | 580 lines, 28 collected tests | route/api, db/session, security, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_review_regressions.py` | 930 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_rename_user_owner_sync.py` | 686 lines, 26 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_helpers_import_state.py` | 426 lines, 26 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_taxonomy.py` | 26 collected tests | security, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_tool_path_confinement.py` | 24 collected tests | import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_copilot.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_research_utils.py` | 23 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_api_chat_security.py` | 401 lines, 22 collected tests | route/api, db/session, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_tool_support_heuristic.py` | 22 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_platform_compat.py` | 318 lines, 21 collected tests | import-state, filesystem, subprocess/script | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_context_compactor.py` | 21 collected tests | db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_prompt_security.py` | 21 collected tests | No obvious setup signals from static scan. | Good first manual-review candidate if test themes are cohesive. |
 | `tests/test_null_owner_gates.py` | 342 lines, 20 collected tests | route/api, db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_youtube_handler_consolidation.py` | 20 collected tests | route/api, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_calendar_recurrence.py` | 338 lines | No obvious setup signals from static scan. | Plan split boundaries before editing. |
 | `tests/test_workspace_confine.py` | 328 lines | route/api, filesystem, subprocess/script, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_api_token_routes.py` | 578 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_companion_readonly.py` | 402 lines | db/session, import-state | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_set_admin.py` | 317 lines | route/api, import-state, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_imap_leak_fixes.py` | 404 lines | route/api, db/session, security, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_auth_regressions.py` | 375 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_agent_migration_manifest.py` | 340 lines | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_diffusion_server_security.py` | 325 lines | route/api, import-state, security, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_tool_policy.py` | 330 lines | import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_endpoint_owner_scope_followup.py` | 414 lines | route/api, db/session, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_upload_routes_owner_scope.py` | 315 lines | route/api, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_email_owner_scope.py` | 474 lines | route/api, db/session, filesystem, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_upload_handler_atomicity.py` | 401 lines | filesystem, async/threading | Plan split boundaries before editing. |
 | `tests/test_kv_cache_invalidation_2927.py` | 463 lines | route/api, db/session, import-state, async/threading | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_calendar_owner_scope.py` | 345 lines | route/api, db/session, import-state, filesystem, async/threading, ui/static | Defer mechanical split until setup/risk boundaries are mapped. |
 | `tests/test_skills_manager_owner_isolation.py` | 306 lines | import-state, filesystem | Defer mechanical split until setup/risk boundaries are mapped. |
 ## Taxonomy coverage gaps among split candidates
 `uncategorized` is a current taxonomy area, not a builder failure.
 This plan does not reclassify tests because taxonomy changes should be reviewed separately from oversized-file split planning.
 Before using any of these files as a split target, first decide whether the taxonomy should be refined in a separate focused issue/PR.
 | File | Lines | Collected tests | Sub-area | Signals | Suggested follow-up |
 |---|---:|---:|---|---|---|
 | `tests/test_pr_blocker_audit.py` | 964 | 58 | pr_blocker_audit | import-state, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_agent_loop.py` | 469 | 52 | agent_loop | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_service_health.py` | 472 | 47 | service_health | async/threading | Review taxonomy mapping before using as a split target. |
 | `tests/test_run_focus.py` | 399 | 47 | run_focus | security, filesystem, subprocess/script, ui/static | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_endpoint_probing.py` | 411 | 34 | endpoint_probing | route/api, db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_chat_helpers.py` | 264 | 31 | chat_helpers | route/api | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_model_context.py` | 251 | 30 | model_context | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_endpoint_resolver.py` | 148 | 30 | endpoint_resolver | - | Review taxonomy mapping before using as a split target. |
 | `tests/test_upload_limits_centralized.py` | 110 | 29 | upload_limits_centralized | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_review_regressions.py` | 930 | 26 | review_regressions | route/api, db/session, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_taxonomy.py` | 145 | 26 | taxonomy | security, ui/static | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_copilot.py` | 170 | 23 | copilot | - | Review taxonomy mapping before using as a split target. |
 | `tests/test_tool_support_heuristic.py` | 166 | 22 | tool_support_heuristic | - | Review taxonomy mapping before using as a split target. |
 | `tests/test_platform_compat.py` | 318 | 21 | platform_compat | import-state, filesystem, subprocess/script | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_context_compactor.py` | 233 | 21 | context_compactor | db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_youtube_handler_consolidation.py` | 104 | 20 | youtube_handler_consolidation | route/api, import-state | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_workspace_confine.py` | 328 | 18 | workspace_confine | route/api, filesystem, subprocess/script, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_companion_readonly.py` | 402 | 17 | companion_readonly | db/session, import-state | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_set_admin.py` | 317 | 17 | set_admin | route/api, import-state, filesystem, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_imap_leak_fixes.py` | 404 | 15 | imap_leak_fixes | route/api, db/session, security, filesystem | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_agent_migration_manifest.py` | 340 | 15 | agent_migration_manifest | import-state, filesystem | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_tool_policy.py` | 330 | 13 | tool_policy | import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 | `tests/test_upload_handler_atomicity.py` | 401 | 9 | upload_handler_atomicity | filesystem, async/threading | Review taxonomy mapping before using as a split target. |
 | `tests/test_kv_cache_invalidation_2927.py` | 463 | 8 | kv_cache_invalidation_2927 | route/api, db/session, import-state, async/threading | Review taxonomy and setup/risk boundaries before any split. |
 ## Suggested first manual-review candidates
 These are not automatic split approvals. They are categorized candidates with enough size/collection value and no route/API, DB/session, import-state, or security signal from the static scan.
 Files still in the `uncategorized` taxonomy area are listed separately below so taxonomy review does not get mixed into the first split decision.
 | File | Lines | Collected tests | Area | Sub-area | Signals | Why this is a candidate |
 |---|---:|---:|---|---|---|---|
 | `tests/test_provider_classification.py` | 188 | 67 | services | provider | - | 67 collected tests |
 | `tests/test_provider_endpoints.py` | 241 | 58 | services | provider | subprocess/script | 58 collected tests |
 | `tests/test_llm_core_temperature.py` | 196 | 41 | services | llm | - | 41 collected tests |
 | `tests/test_provider_detection.py` | 148 | 31 | services | provider | - | 31 collected tests |
 | `tests/test_embedding_lanes.py` | 1104 | 29 | services | embedding | filesystem | 1104 lines, 29 collected tests |
 | `tests/test_research_utils.py` | 97 | 23 | services | research | - | 23 collected tests |
 | `tests/test_prompt_security.py` | 203 | 21 | security | security | - | 21 collected tests |
 | `tests/test_calendar_recurrence.py` | 338 | 19 | services | calendar | - | 338 lines |
 ## High-risk candidates to defer first
 These files may still be split later, but not as the first implementation slice without a separate manual boundary review.
 | File | Lines | Collected tests | High-risk signals |
 |---|---:|---:|---|
 | `tests/test_model_routes.py` | 1778 | 139 | db/session, import-state, route/api |
 | `tests/test_security_regressions.py` | 1224 | 92 | db/session, import-state, route/api, security |
 | `tests/test_cookbook_helpers.py` | 912 | 65 | route/api |
 | `tests/test_shell_routes.py` | 481 | 63 | import-state, route/api |
 | `tests/test_pr_blocker_audit.py` | 964 | 58 | import-state, security |
 | `tests/test_agent_loop.py` | 469 | 52 | db/session, import-state |
 | `tests/test_run_focus.py` | 399 | 47 | security |
 | `tests/test_endpoint_probing.py` | 411 | 34 | db/session, import-state, route/api |
 | `tests/test_llm_core_anthropic_temp_omit.py` | 94 | 32 | db/session |
 | `tests/test_chat_helpers.py` | 264 | 31 | route/api |
 | `tests/test_model_context.py` | 251 | 30 | db/session, import-state |
 | `tests/test_upload_limits_centralized.py` | 110 | 29 | import-state |
 | `tests/test_email_oauth.py` | 580 | 28 | db/session, route/api, security |
 | `tests/test_review_regressions.py` | 930 | 26 | db/session, import-state, route/api |
 | `tests/test_rename_user_owner_sync.py` | 686 | 26 | db/session, import-state, route/api |
 ## Rules for future split PRs
 - One file or one coherent file-family per PR.
 - No assertion rewrites mixed with file moves.
 - No helper extraction mixed with file moves.
 - No production code changes.
 - No CI workflow changes.
 - Preserve existing markers and taxonomy unless the split issue explicitly says otherwise.
 - Validate the original file's collected tests before and after the split.
 - Validate any neighboring taxonomy/focused-runner behavior if paths change.
 - Treat files with route/API, DB/session, import-state, or security signals as higher-risk until manually reviewed.
 ## Suggested next step
 Use this plan to choose the first actual oversized-file split issue.
 The first split should prefer a file with high review value and low setup risk.
 Do not start a split PR from this planning issue alone if the file's boundaries are still ambiguous.
 ## Reproduction command
 This document was generated with:
 ```bash
 .venv/bin/python tests/tools/build_oversized_test_split_plan.py
 ```
 ## Freshness check
 After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:
 ```bash
 .venv/bin/python tests/tools/build_oversized_test_split_plan.py
 git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md
 ```
@@ -0,0 +1,104 @@
 from src import ai_interaction
 class _GenerationResponse:
    status_code = 200
    text = ""
    def __init__(self, image_url):
        self._image_url = image_url
    def json(self):
        return {"data": [{"url": self._image_url}]}
 class _DownloadResponse:
    status_code = 503
    content = b""
 def _patch_generation(monkeypatch, image_url):
    async def _post(self, url, json, headers):
        return _GenerationResponse(image_url)
    class _AsyncClient:
        def __init__(self, *args, **kwargs):
            pass
        async def __aenter__(self):
            return self
        async def __aexit__(self, *exc):
            return False
        post = _post
    import httpx
    import src.settings as settings
    monkeypatch.setattr(settings, "load_settings", lambda: {})
    monkeypatch.setattr(httpx, "AsyncClient", _AsyncClient)
    monkeypatch.setattr(
        ai_interaction,
        "_resolve_model",
        lambda model_spec, owner=None: (
            "https://api.openai.example/v1/chat/completions",
            "dall-e-3",
            {"Authorization": "Bearer test"},
        ),
    )
 async def test_generate_image_validates_provider_url_before_download(monkeypatch):
    import httpx
    import src.url_safety as url_safety
    provider_url = "https://images.example.com/generated.png?sig=abc"
    events = []
    _patch_generation(monkeypatch, provider_url)
    def _check_outbound_url(url, *, block_private=False):
        events.append(("check", url, block_private))
        return True, "ok"
    def _get(url, *, timeout):
        events.append(("get", url, timeout))
        return _DownloadResponse()
    monkeypatch.setattr(url_safety, "check_outbound_url", _check_outbound_url)
    monkeypatch.setattr(httpx, "get", _get)
    result = await ai_interaction.do_generate_image("draw a chair\ndall-e-3")
    assert result["image_url"] == provider_url
    assert events == [
        ("check", provider_url, False),
        ("get", provider_url, 60),
    ]
 async def test_generate_image_rejects_unsafe_provider_url_without_download(monkeypatch):
    import httpx
    import src.url_safety as url_safety
    unsafe_url = "http://169.254.169.254/latest/meta-data"
    events = []
    _patch_generation(monkeypatch, unsafe_url)
    def _check_outbound_url(url, *, block_private=False):
        events.append(("check", url, block_private))
        return False, "link-local address blocked (SSRF metadata risk): 169.254.169.254"
    def _get(url, *, timeout):
        raise AssertionError("unsafe provider image URL must not be downloaded")
    monkeypatch.setattr(url_safety, "check_outbound_url", _check_outbound_url)
    monkeypatch.setattr(httpx, "get", _get)
    result = await ai_interaction.do_generate_image("draw a chair\ndall-e-3")
    assert result["error"] == (
        "Image API returned unsafe image URL: "
        "link-local address blocked (SSRF metadata risk): 169.254.169.254"
    )
    assert events == [("check", unsafe_url, False)]
@@ -502,3 +502,77 @@ def test_delete_token_owner_check_skipped_when_auth_disabled(monkeypatch, token_
    resp = delete_token(request=req, token_id="tok123")
    assert resp == {"status": "deleted"}
    fake_session.delete.assert_called_once_with(fake_token)
 # ---------------------------------------------------------------------------
 # 7. PATCH /api/tokens/{id} — non-object JSON bodies must not 500
 # ---------------------------------------------------------------------------
 def test_update_token_with_array_body_does_not_500(monkeypatch, token_routes_mod):
    """PATCH body of [] must be normalised to {} and not raise."""
    monkeypatch.setenv("AUTH_ENABLED", "true")
    mod = token_routes_mod
    token = SimpleNamespace(
        id="tok123", name="original", owner="alice",
        token_prefix="ody_orig", scopes="email:read", is_active=True,
    )
    fake_session = MagicMock()
    fake_session.query.return_value.filter.return_value.first.return_value = token
    monkeypatch.setattr(mod, "get_db_session", lambda: _db_ctx(fake_session))
    invalidator = MagicMock()
    req = _patch_request(invalidator, [])
    update_token = _get_handler(mod, "PATCH", "/tokens/{token_id}")
    resp = asyncio.run(update_token(request=req, token_id="tok123"))
    # Name and scopes must be unchanged — payload was normalised to {}
    assert token.name == "original"
    assert token.scopes == "email:read"
    assert resp["name"] == "original"
 def test_update_token_with_null_body_does_not_500(monkeypatch, token_routes_mod):
    """PATCH body of null must be normalised to {} and not raise."""
    monkeypatch.setenv("AUTH_ENABLED", "true")
    mod = token_routes_mod
    token = SimpleNamespace(
        id="tok123", name="original", owner="alice",
        token_prefix="ody_orig", scopes="chat", is_active=True,
    )
    fake_session = MagicMock()
    fake_session.query.return_value.filter.return_value.first.return_value = token
    monkeypatch.setattr(mod, "get_db_session", lambda: _db_ctx(fake_session))
    invalidator = MagicMock()
    req = _patch_request(invalidator, None)
    update_token = _get_handler(mod, "PATCH", "/tokens/{token_id}")
    resp = asyncio.run(update_token(request=req, token_id="tok123"))
    assert token.name == "original"
    assert token.scopes == "chat"
 def test_update_token_normal_object_still_works(monkeypatch, token_routes_mod):
    """Normal dict payload continues to update fields as before."""
    monkeypatch.setenv("AUTH_ENABLED", "true")
    mod = token_routes_mod
    token = SimpleNamespace(
        id="tok123", name="original", owner="alice",
        token_prefix="ody_orig", scopes="email:read", is_active=True,
    )
    fake_session = MagicMock()
    fake_session.query.return_value.filter.return_value.first.return_value = token
    monkeypatch.setattr(mod, "get_db_session", lambda: _db_ctx(fake_session))
    invalidator = MagicMock()
    req = _patch_request(invalidator, {"name": "updated"})
    update_token = _get_handler(mod, "PATCH", "/tokens/{token_id}")
    resp = asyncio.run(update_token(request=req, token_id="tok123"))
    assert token.name == "updated"
    assert resp["name"] == "updated"
    invalidator.assert_called_once()
@@ -0,0 +1,88 @@
 """do_manage_calendar must honour abbreviated reminder phrasings like "mins"/"hrs".
 `_reminder_minutes` parsed the reminder offset with regexes anchored on
 `(?:m|min|minute|minutes)\b` / `(?:h|hr|hour|hours)\b`. The trailing `\b`
 made the very common plural abbreviations "mins" and "hrs" fail to match
 (after "min" the next char "s" is a word char, so no boundary), so a request
 like ``reminder_minutes: "5 mins"`` silently produced no reminder at all —
 even though the sibling duration parser (no `\b`) already accepted them.
 """
 import json
 import sys
 import uuid
 import pytest
 from tests.helpers.import_state import clear_fake_database_modules
 from tests.helpers.sqlite_db import make_temp_sqlite
 clear_fake_database_modules()
 import core.database as cdb
 from core.database import Note
 _TS, _ENGINE, _TMPDB = make_temp_sqlite(cdb.Base.metadata)
@pytest.fixture(autouse=True)
 def _bind_temp_db(monkeypatch):
    monkeypatch.setitem(sys.modules, "core.database", cdb)
    parent = sys.modules.get("core")
    if parent is not None:
        monkeypatch.setattr(parent, "database", cdb, raising=False)
    monkeypatch.setattr(cdb, "SessionLocal", _TS)
    yield
 async def _create_with_reminder(reminder, owner):
    from src.tool_implementations import do_manage_calendar
    payload = {
        "action": "create_event",
        "summary": "Dentist",
        # Far-future so the reminder is never "already passed".
        "dtstart": "2030-01-01T10:00:00",
        "reminder_minutes": reminder,
    }
    return await do_manage_calendar(json.dumps(payload), owner=owner)
@pytest.mark.parametrize("reminder,expected", [
    ("5 mins", 5),
    ("10 mins", 10),
    ("2 hrs", 120),
    ("1 hr", 60),
    ("15 minutes", 15),   # regression: long form still works
    ("30m", 30),          # regression: bare unit still works
 ])
 async def test_reminder_minutes_accepts_abbreviations(reminder, expected):
    owner = "tester-" + uuid.uuid4().hex[:6]
    res = await _create_with_reminder(reminder, owner)
    assert res.get("exit_code") == 0, res
    assert f"reminder {expected} min before" in res.get("response", ""), res
    db = _TS()
    try:
        note = (
            db.query(Note)
            .filter(Note.owner == owner, Note.title == "Reminder: Dentist")
            .first()
        )
        assert note is not None, "reminder note should have been created"
    finally:
        db.close()
 async def test_no_reminder_when_offset_absent():
    owner = "tester-" + uuid.uuid4().hex[:6]
    from src.tool_implementations import do_manage_calendar
    payload = {
        "action": "create_event",
        "summary": "No Reminder Event",
        "dtstart": "2030-02-01T10:00:00",
    }
    res = await do_manage_calendar(json.dumps(payload), owner=owner)
    assert res.get("exit_code") == 0, res
    assert "reminder set" not in res.get("response", ""), res
@@ -30,7 +30,7 @@ class _Session:
 def test_allowed_models_legacy_empty_list_remains_unrestricted(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    _enforce_chat_privileges(
        _Request({"allowed_models": [], "max_messages_per_day": 0}),
@@ -39,7 +39,7 @@ def test_allowed_models_legacy_empty_list_remains_unrestricted(monkeypatch):
 def test_allowed_models_explicit_empty_restricted_list_blocks_all_models(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    with pytest.raises(HTTPException) as exc:
        _enforce_chat_privileges(
@@ -56,7 +56,7 @@ def test_allowed_models_explicit_empty_restricted_list_blocks_all_models(monkeyp
 def test_allowed_models_nonempty_list_still_restricts_without_new_flag(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    _enforce_chat_privileges(
        _Request({"allowed_models": ["provider/model-a"], "max_messages_per_day": 0}),
@@ -70,7 +70,7 @@ def test_allowed_models_nonempty_list_still_restricts_without_new_flag(monkeypat
 def test_no_restriction_allows_any_model(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    privs = {"allowed_models": [], "block_all_models": False, "max_messages_per_day": 0}
    _enforce_chat_privileges(_Request(privs), _Session("provider/model-a"))
@@ -78,7 +78,7 @@ def test_no_restriction_allows_any_model(monkeypatch):
 def test_specific_allowlist_blocks_models_outside_it(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    privs = {
        "allowed_models": ["gpt-4"],
@@ -92,7 +92,7 @@ def test_specific_allowlist_blocks_models_outside_it(monkeypatch):
 def test_block_all_models_blocks_regardless_of_allowed_models_contents(monkeypatch):
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "alice")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "alice")
    # Even if allowed_models contains entries, block_all_models wins.
    privs = {
@@ -111,7 +111,7 @@ def test_block_all_models_blocks_regardless_of_allowed_models_contents(monkeypat
 def test_admin_user_is_never_blocked(monkeypatch):
    from core.auth import ADMIN_PRIVILEGES
-    monkeypatch.setattr("routes.chat_helpers.get_current_user", lambda request: "admin")
+    monkeypatch.setattr("routes.chat_helpers.effective_user", lambda request: "admin")
    class _AdminAuthManager:
        def get_privileges(self, username):
@@ -7,12 +7,39 @@ in ``remoteHost`` would be injected into that command.
 These pin validation on the host/port before they reach the ssh string, matching
 the validators the rest of the cookbook routes already apply.
 """
 import asyncio
 import pytest
 from fastapi import HTTPException
 from starlette.requests import Request
 import routes.codex_routes as codex_routes
 def _route_endpoint(path: str, method: str):
    router = codex_routes.setup_codex_routes()
    for route in router.routes:
        if route.path == path and method in route.methods:
            return route.endpoint
    raise AssertionError(f"{method} {path} route not found")
 def _launch_request() -> Request:
    request = Request(
        {
            "type": "http",
            "method": "POST",
            "path": "/api/codex/cookbook/adopt",
            "headers": [],
            "state": {},
        }
    )
    request.state.api_token = True
    request.state.api_token_owner = "alice"
    request.state.api_token_scopes = ["cookbook:launch"]
    return request
 def test_rejects_remote_host_with_shell_metacharacters():
    task = {"remoteHost": "box; rm -rf ~", "sshPort": ""}
    with pytest.raises(HTTPException) as exc:
@@ -47,3 +74,26 @@ def test_default_ssh_port_omits_flag():
    )
    assert host == "box"
    assert port_flag == ""
 def test_adopt_rejects_ssh_option_host_before_shell(monkeypatch):
    calls = []
    async def fail_if_shell_runs(*args, **kwargs):
        calls.append((args, kwargs))
        raise RuntimeError("shell should not run for invalid host")
    monkeypatch.setattr(asyncio, "create_subprocess_shell", fail_if_shell_runs)
    endpoint = _route_endpoint("/api/codex/cookbook/adopt", "POST")
    body = {
        "tmux_session": "serve_abc123",
        "model": "org/model",
        "host": "-oProxyCommand=sh",
    }
    with pytest.raises(HTTPException) as exc:
        asyncio.run(endpoint(_launch_request(), body))
    assert exc.value.status_code == 400
    assert calls == []
@@ -13,6 +13,9 @@ import json
 from types import SimpleNamespace
 from unittest.mock import MagicMock
 import pytest
 from fastapi import HTTPException
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 # core.database instantiates SQLAlchemy declarative classes at import time, which
@@ -225,12 +228,34 @@ def test_models_route_scopes_api_token_to_token_owner(monkeypatch):
    endpoints = _call_models_route(
        monkeypatch,
        rows,
-        _request(api_token=True, api_token_owner="alice", current_user="api"),
+        _request(
            api_token=True,
            api_token_owner="alice",
            api_token_scopes=["chat"],
            current_user="api",
        ),
    )
    assert _endpoint_names(endpoints) == ["alice-endpoint", "shared-endpoint"]
 def test_models_route_rejects_api_token_without_chat_scope(monkeypatch):
    monkeypatch.setattr(companion_routes, "get_current_user", lambda request: "api")
    with pytest.raises(HTTPException) as exc:
        _models_route()(
            _request(
                api_token=True,
                api_token_owner="alice",
                api_token_scopes=["todos:read"],
                current_user="api",
            )
        )
    assert exc.value.status_code == 403
    assert "chat scope" in exc.value.detail
 def test_models_route_unresolved_owner_returns_only_shared_rows(monkeypatch):
    rows = [
        _ep(1, "alice-endpoint", "alice"),
@@ -242,7 +267,12 @@ def test_models_route_unresolved_owner_returns_only_shared_rows(monkeypatch):
    endpoints = _call_models_route(
        monkeypatch,
        rows,
-        _request(api_token=True, api_token_owner=None, current_user="api"),
+        _request(
            api_token=True,
            api_token_owner=None,
            api_token_scopes=["chat"],
            current_user="api",
        ),
    )
    assert _endpoint_names(endpoints) == ["shared-endpoint"]
@@ -786,6 +786,50 @@ def test_cached_model_scan_reports_plain_dir_gguf(tmp_path):
    assert ggufs[3]["quant"] == "BF16"
 def test_cached_model_scan_uses_ollama_api_before_cli_and_windows_opt_in():
    script = _cached_model_scan_script()
    assert "scan_ollama_api()\nscan_ollama()" in script
    assert "if any(m.get('is_ollama') for m in models): return" in script
    assert "os.name == 'nt'" in script
    assert "ODYSSEUS_ALLOW_OLLAMA_CLI_SCAN" in script
@pytest.mark.skipif(os.name != "nt", reason="Windows Ollama CLI startup guard")
 def test_cached_model_scan_does_not_launch_ollama_cli_on_windows(tmp_path):
    """Official Ollama for Windows can auto-start the tray/server on `ollama list`.
    The read-only cache scanner must not invoke that CLI unless explicitly opted in.
    """
    marker = tmp_path / "ollama-called.txt"
    fake_ollama = tmp_path / "ollama.cmd"
    fake_ollama.write_text(
        "@echo off\r\n"
        f'echo called>"{marker}"\r\n'
        "echo NAME ID SIZE MODIFIED\r\n"
        "echo local-model:latest abc 1 GB now\r\n",
        encoding="utf-8",
    )
    empty_home = tmp_path / "home"
    empty_home.mkdir()
    scan_py = tmp_path / "scan_cache.py"
    scan_py.write_text(_cached_model_scan_script(), encoding="utf-8")
    env = dict(os.environ)
    env["PATH"] = str(tmp_path) + os.pathsep + env.get("PATH", "")
    env["HOME"] = str(empty_home)
    env.pop("ODYSSEUS_ALLOW_OLLAMA_CLI_SCAN", None)
    proc = subprocess.run(
        [sys.executable, str(scan_py)],
        check=True,
        capture_output=True,
        text=True,
        env=env,
    )
    assert marker.exists() is False
    assert all(m.get("backend") != "ollama" for m in json.loads(proc.stdout))
 def test_cached_model_scan_uses_huggingface_cache_env(tmp_path):
    """Docker recreates can leave the persisted HF cache outside HOME.
    The Serve scanner should honor the cache env path instead of only ~/.cache.
@@ -0,0 +1,50 @@
 import json
 import pytest
 from src import cookbook_serve_lifecycle as lifecycle
@pytest.mark.asyncio
 async def test_tick_persists_only_successfully_stopped_serves(tmp_path, monkeypatch):
    state_path = tmp_path / "cookbook_state.json"
    state_path.write_text(
        json.dumps({
            "tasks": [
                {
                    "id": "stop-succeeds",
                    "type": "serve",
                    "status": "running",
                    "_scheduledStopAtMs": 0,
                },
                {
                    "id": "stop-fails",
                    "type": "serve",
                    "status": "running",
                    "_scheduledStopAtMs": 0,
                },
            ]
        }),
        encoding="utf-8",
    )
    async def fake_stop_serve(session_id, remote_host="", ssh_port=""):
        return session_id == "stop-succeeds"
    async def fake_delete_endpoint(task):
        return None
    monkeypatch.setattr(lifecycle, "COOKBOOK_STATE_FILE", str(state_path))
    monkeypatch.setattr(lifecycle, "_stop_serve", fake_stop_serve)
    monkeypatch.setattr(lifecycle, "_delete_endpoint_for_task", fake_delete_endpoint)
    await lifecycle._tick()
    tasks = {
        task["id"]: task
        for task in json.loads(state_path.read_text(encoding="utf-8"))["tasks"]
    }
    assert tasks["stop-succeeds"]["status"] == "stopped"
    assert tasks["stop-succeeds"]["_scheduledStopAtMs"] is None
    assert tasks["stop-fails"]["status"] == "running"
    assert tasks["stop-fails"]["_scheduledStopAtMs"] == 0
@@ -0,0 +1,580 @@
 """Tests for the Google OAuth2 email helpers.
 Covers the security-critical surface added for Google Workspace / .edu
 IMAP/SMTP support:
 - `make_oauth_state` / `verify_oauth_state` — HMAC-signed OAuth state so the
  callback can't be CSRF'd or have its account_id/owner tampered with.
 - `_smtp_ready` — an OAuth account (no stored password) must still count as
  send-capable; a host+user-only account without password or OAuth must not.
 - `_xoauth2_raw` / `_xoauth2_bytes` — SASL XOAUTH2 framing for SMTP/IMAP.
 - `_refresh_google_token` — token refresh stores result encrypted; failure is
  silent (no token/secret in logs or return value).
 - `_get_valid_google_token` — uses cached token when fresh; calls refresh when
  expired.
 - `google_oauth_callback` (real route) — invalid/tampered/missing state and
  provider errors return generic redirects with no PII; owner mismatch refuses
  the token write; a valid owner writes encrypted tokens only to the intended
  account.
 - `list_email_accounts` (real route) — exposes OAuth status but never token
  values.
 - `_imap_connect` — password accounts use login(); OAuth accounts use XOAUTH2.
 Route tests pull the live endpoint out of `setup_email_routes()` and call it
 directly — they pin the real handler, not a re-implementation. The ASGI app is
 not booted; outbound HTTP is mocked and the DB is an isolated in-memory SQLite.
 """
 import base64
 import json
 import time
 import unittest.mock as mock
 import pytest
 # ── OAuth state signing ──────────────────────────────────────────
 def test_oauth_state_round_trips_account_and_owner():
    from routes.email_helpers import make_oauth_state, verify_oauth_state
    state = make_oauth_state("acct-123", "user@example.com")
    payload = verify_oauth_state(state)
    assert payload is not None
    assert payload["a"] == "acct-123"
    assert payload["o"] == "user@example.com"
    assert payload["n"]  # nonce present
 def test_oauth_state_nonce_is_unique_per_call():
    from routes.email_helpers import make_oauth_state, verify_oauth_state
    a = verify_oauth_state(make_oauth_state("acct", "o"))
    b = verify_oauth_state(make_oauth_state("acct", "o"))
    assert a["n"] != b["n"]
 def test_oauth_state_rejects_tampered_account_id():
    from routes.email_helpers import make_oauth_state, verify_oauth_state
    state = make_oauth_state("acct-123", "user@example.com")
    decoded = base64.urlsafe_b64decode(state.encode()).decode()
    payload_str, sig = decoded.rsplit("|", 1)
    payload = json.loads(payload_str)
    payload["a"] = "evil-acct"  # attacker swaps the target account
    forged = base64.urlsafe_b64encode(
        (json.dumps(payload, separators=(",", ":")) + "|" + sig).encode()
    ).decode()
    assert verify_oauth_state(forged) is None
 def test_oauth_state_rejects_forged_signature():
    from routes.email_helpers import make_oauth_state, verify_oauth_state
    state = make_oauth_state("acct-123", "user@example.com")
    decoded = base64.urlsafe_b64decode(state.encode()).decode()
    payload_str, _ = decoded.rsplit("|", 1)
    forged = base64.urlsafe_b64encode((payload_str + "|" + "deadbeef" * 8).encode()).decode()
    assert verify_oauth_state(forged) is None
@pytest.mark.parametrize("garbage", ["", "not-base64-at-all", "###", "a|b|c"])
 def test_oauth_state_rejects_garbage(garbage):
    from routes.email_helpers import verify_oauth_state
    assert verify_oauth_state(garbage) is None
 # ── _smtp_ready: OAuth accounts have no password but can still send ──
 def test_smtp_ready_true_for_oauth_account_without_password():
    from routes.email_routes import _smtp_ready
    cfg = {
        "smtp_host": "smtp.gmail.com",
        "smtp_user": "me@nyu.edu",
        "smtp_password": "",
        "oauth_provider": "google",
    }
    assert _smtp_ready(cfg) is True
 def test_smtp_ready_true_for_password_account():
    from routes.email_routes import _smtp_ready
    cfg = {
        "smtp_host": "smtp.example.com",
        "smtp_user": "me@example.com",
        "smtp_password": "app-password",
        "oauth_provider": "",
    }
    assert _smtp_ready(cfg) is True
 def test_smtp_ready_false_without_password_or_oauth():
    from routes.email_routes import _smtp_ready
    cfg = {
        "smtp_host": "smtp.example.com",
        "smtp_user": "me@example.com",
        "smtp_password": "",
        "oauth_provider": "",
    }
    assert _smtp_ready(cfg) is False
 def test_smtp_ready_false_without_host():
    from routes.email_routes import _smtp_ready
    cfg = {"smtp_host": "", "smtp_user": "me@x.com", "oauth_provider": "google"}
    assert _smtp_ready(cfg) is False
 # ── XOAUTH2 SASL framing ─────────────────────────────────────────
 def test_xoauth2_raw_is_unencoded_sasl_frame():
    from routes.email_helpers import _xoauth2_raw
    assert _xoauth2_raw("me@nyu.edu", "tok123") == "user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
 def test_xoauth2_bytes_is_raw_frame_encoded():
    from routes.email_helpers import _xoauth2_bytes
    assert _xoauth2_bytes("me@nyu.edu", "tok123") == b"user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
 # ── Helpers for in-memory DB fixtures ────────────────────────────
 def _make_db():
    """Return (Session, SessionFactory) backed by an isolated in-memory SQLite DB.
    Used to test DB-touching helpers without the real database.
    The factory lets tests open a fresh session after the helper closes its own.
    """
    from sqlalchemy import create_engine
    from sqlalchemy.orm import sessionmaker
    from core.database import Base
    engine = create_engine("sqlite:///:memory:", connect_args={"check_same_thread": False})
    Base.metadata.create_all(engine)
    Factory = sessionmaker(bind=engine)
    return Factory(), Factory
 def _make_account(session, account_id="acct-1", owner="alice", **kwargs):
    """Insert a minimal EmailAccount row and return it."""
    from core.database import EmailAccount
    row = EmailAccount(
        id=account_id,
        owner=owner,
        name=kwargs.get("name", "Test"),
        from_address=kwargs.get("from_address", "test@example.com"),
        imap_host=kwargs.get("imap_host", "imap.gmail.com"),
        imap_port=kwargs.get("imap_port", 993),
        imap_user=kwargs.get("imap_user", "test@example.com"),
        smtp_host=kwargs.get("smtp_host", "smtp.gmail.com"),
        smtp_port=kwargs.get("smtp_port", 587),
        smtp_user=kwargs.get("smtp_user", "test@example.com"),
    )
    for k, v in kwargs.items():
        if hasattr(row, k):
            setattr(row, k, v)
    session.add(row)
    session.commit()
    return row
 # ── Token encryption at rest ─────────────────────────────────────
 def test_refresh_token_stored_encrypted_not_raw():
    """_refresh_google_token must encrypt the new access token before writing it
    to the DB — storing the raw token string would expose credentials at rest."""
    from src.secret_storage import encrypt as _enc, decrypt as _dec
    from core.database import EmailAccount
    raw_token = "ya29.test_access_token_raw"
    db, Factory = _make_db()
    _make_account(db, account_id="acct-r", owner="bob",
                  oauth_refresh_token=_enc("refresh-tok-xyz"))
    db.close()
    fake_resp = mock.MagicMock()
    fake_resp.raise_for_status = mock.MagicMock()
    fake_resp.json.return_value = {"access_token": raw_token, "expires_in": 3600}
    with mock.patch("httpx.post", return_value=fake_resp), \
         mock.patch("core.database.SessionLocal", Factory), \
         mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
             "GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
         }.get(k, d)):
        from routes.email_helpers import _refresh_google_token
        result = _refresh_google_token("acct-r")
    verify_db = Factory()
    row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-r").first()
    stored = row.oauth_access_token
    verify_db.close()
    assert result == raw_token, "function should return the plain access token to callers"
    assert stored != raw_token, "raw token must not be stored directly in the DB"
    assert _dec(stored) == raw_token, "stored value must decrypt back to the raw token"
 def test_refresh_stores_encrypted_expiry_not_token():
    """oauth_token_expiry stores only a timestamp, never the token value."""
    from src.secret_storage import encrypt as _enc
    from core.database import EmailAccount
    db, Factory = _make_db()
    _make_account(db, account_id="acct-e", owner="bob",
                  oauth_refresh_token=_enc("ref-tok"))
    db.close()
    fake_resp = mock.MagicMock()
    fake_resp.raise_for_status = mock.MagicMock()
    fake_resp.json.return_value = {"access_token": "ya29.secret", "expires_in": 3600}
    with mock.patch("httpx.post", return_value=fake_resp), \
         mock.patch("core.database.SessionLocal", Factory), \
         mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
             "GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
         }.get(k, d)):
        from routes.email_helpers import _refresh_google_token
        _refresh_google_token("acct-e")
    verify_db = Factory()
    row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-e").first()
    expiry = row.oauth_token_expiry
    verify_db.close()
    assert "ya29" not in (expiry or ""), \
        "token_expiry must be a timestamp, not the token string"
 # ── Real OAuth callback route ─────────────────────────────────────
 #
 # These pull the actual google_oauth_callback endpoint out of the router and
 # invoke it — they pin the real route's behaviour, not a re-implementation, so
 # they fail if the ownership/state guards are ever removed or weakened.
 def _callback_endpoint():
    """Return the live google_oauth_callback endpoint from the email router."""
    from routes.email_routes import setup_email_routes
    router = setup_email_routes()
    for route in router.routes:
        if route.path == "/api/email/oauth/google/callback" and "GET" in getattr(route, "methods", set()):
            return route.endpoint
    raise AssertionError("google_oauth_callback route not found")
 class _FakeRequest:
    """Minimal stand-in for starlette Request — the callback only reads headers."""
    headers = {"host": "localhost:7000"}
 def _location(resp):
    """Pull the redirect target out of a RedirectResponse."""
    return resp.headers["location"]
@pytest.mark.asyncio
 async def test_callback_missing_code_returns_generic_error():
    """No `code` query param → generic error redirect, with no account id, owner,
    or state echoed back into the URL."""
    from routes.email_helpers import make_oauth_state
    callback = _callback_endpoint()
    state = make_oauth_state("acct-1", "alice")
    resp = await callback(code=None, state=state, error=None, request=_FakeRequest())
    loc = _location(resp)
    assert "email_oauth_error=missing_code" in loc
    assert "acct-1" not in loc, "account id must not appear in redirect URL"
    assert "alice" not in loc, "owner must not appear in redirect URL"
@pytest.mark.asyncio
 async def test_callback_provider_error_returns_generic_error():
    """An `error` from Google → generic error redirect, no raw provider text."""
    callback = _callback_endpoint()
    resp = await callback(code=None, state=None, error="access_denied", request=_FakeRequest())
    loc = _location(resp)
    assert "email_oauth_error=google_error" in loc
    assert "access_denied" not in loc, "raw provider error must not leak into redirect"
@pytest.mark.asyncio
 async def test_callback_tampered_state_returns_generic_error_no_leak():
    """Tampered/invalid state → invalid_state redirect; the auth code and any
    token must never appear in the redirect URL."""
    callback = _callback_endpoint()
    resp = await callback(code="4/secret-auth-code", state="not-a-valid-state",
                          error=None, request=_FakeRequest())
    loc = _location(resp)
    assert "email_oauth_error=invalid_state" in loc
    assert "4/secret-auth-code" not in loc, "auth code must not leak into redirect"
    assert "token" not in loc
@pytest.mark.asyncio
 async def test_callback_owner_mismatch_does_not_write_tokens():
    """A signed, valid state whose owner does not match the target account's
    owner must NOT write tokens — this blocks one authenticated user from
    binding their Google account onto another user's mailbox row.
    """
    from routes.email_helpers import make_oauth_state
    from core.database import EmailAccount
    db, Factory = _make_db()
    _make_account(db, account_id="acct-x", owner="alice")
    db.close()
    # Token-exchange + userinfo would succeed — the point is the ownership gate
    # rejects the write *before* trusting them.
    token_resp = mock.MagicMock()
    token_resp.raise_for_status = mock.MagicMock()
    token_resp.json.return_value = {"access_token": "ya29.attacker", "refresh_token": "r", "expires_in": 3600}
    userinfo_resp = mock.MagicMock()
    userinfo_resp.is_success = True
    userinfo_resp.json.return_value = {"email": "bob@evil.com", "name": "Bob"}
    # State is genuinely signed, but for owner "bob" — not the row owner "alice".
    state = make_oauth_state("acct-x", "bob")
    with mock.patch("httpx.post", return_value=token_resp), \
         mock.patch("httpx.get", return_value=userinfo_resp), \
         mock.patch("core.database.SessionLocal", Factory):
        callback = _callback_endpoint()
        resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
    loc = _location(resp)
    assert "email_oauth_error=ownership_error" in loc
    verify_db = Factory()
    row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-x").first()
    token_after = row.oauth_access_token
    verify_db.close()
    assert token_after is None, "no token may be written when ownership check fails"
@pytest.mark.asyncio
 async def test_callback_valid_owner_writes_encrypted_tokens_to_intended_account():
    """A signed state whose owner matches the target account writes the tokens —
    and only to that account, stored encrypted (raw token never persisted)."""
    from routes.email_helpers import make_oauth_state
    from src.secret_storage import decrypt as _dec
    from core.database import EmailAccount
    db, Factory = _make_db()
    _make_account(db, account_id="acct-v", owner="alice", imap_host="", smtp_host="")
    _make_account(db, account_id="acct-other", owner="alice")  # must stay untouched
    db.close()
    raw_access = "ya29.legit_access_token"
    raw_refresh = "1//legit_refresh_token"
    token_resp = mock.MagicMock()
    token_resp.raise_for_status = mock.MagicMock()
    token_resp.json.return_value = {"access_token": raw_access, "refresh_token": raw_refresh, "expires_in": 3600}
    userinfo_resp = mock.MagicMock()
    userinfo_resp.is_success = True
    userinfo_resp.json.return_value = {"email": "alice@nyu.edu", "name": "Alice"}
    state = make_oauth_state("acct-v", "alice")
    with mock.patch("httpx.post", return_value=token_resp), \
         mock.patch("httpx.get", return_value=userinfo_resp), \
         mock.patch("core.database.SessionLocal", Factory):
        callback = _callback_endpoint()
        resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
    assert "email_oauth_success=1" in _location(resp)
    verify_db = Factory()
    target = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-v").first()
    other = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-other").first()
    verify_db.close()
    assert target.oauth_provider == "google"
    assert target.oauth_access_token != raw_access, "access token must be stored encrypted"
    assert _dec(target.oauth_access_token) == raw_access
    assert _dec(target.oauth_refresh_token) == raw_refresh
    assert other.oauth_access_token is None, "tokens must only touch the intended account"
 # ── Token refresh scenarios ───────────────────────────────────────
 def test_get_valid_google_token_uses_cached_when_fresh():
    """_get_valid_google_token must NOT call refresh when the stored token is
    still valid (expiry - 60s buffer > now). Refresh is an outbound HTTP call
    that should only happen when genuinely needed."""
    from src.secret_storage import encrypt as _enc
    from routes.email_helpers import _get_valid_google_token
    future_expiry = str(int(time.time()) + 7200)  # 2 hours from now
    cfg = {
        "account_id": "acct-fresh",
        "oauth_access_token": _enc("ya29.fresh_token"),
        "oauth_token_expiry": future_expiry,
    }
    with mock.patch("routes.email_helpers._refresh_google_token") as mock_refresh:
        result = _get_valid_google_token("acct-fresh", cfg)
    assert result == "ya29.fresh_token"
    mock_refresh.assert_not_called()
 def test_get_valid_google_token_refreshes_when_expired():
    """_get_valid_google_token must call refresh when the token is expired."""
    from src.secret_storage import encrypt as _enc
    from routes.email_helpers import _get_valid_google_token
    past_expiry = str(int(time.time()) - 10)  # already expired
    cfg = {
        "account_id": "acct-exp",
        "oauth_access_token": _enc("ya29.old_token"),
        "oauth_token_expiry": past_expiry,
    }
    with mock.patch("routes.email_helpers._refresh_google_token", return_value="ya29.new_token") as mock_refresh:
        result = _get_valid_google_token("acct-exp", cfg)
    mock_refresh.assert_called_once_with("acct-exp")
    assert result == "ya29.new_token"
 def test_refresh_failure_returns_none_no_secret_raised():
    """When the refresh HTTP call fails, _refresh_google_token must return None
    silently. It must not raise an exception or surface token/secret details."""
    from src.secret_storage import encrypt as _enc
    db, Factory = _make_db()
    _make_account(db, account_id="acct-fail", owner="dave",
                  oauth_refresh_token=_enc("ref-tok"))
    db.close()
    failing_resp = mock.MagicMock()
    failing_resp.raise_for_status.side_effect = Exception("401 Unauthorized")
    with mock.patch("httpx.post", return_value=failing_resp), \
         mock.patch("core.database.SessionLocal", Factory), \
         mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
             "GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
         }.get(k, d)):
        from routes.email_helpers import _refresh_google_token
        result = _refresh_google_token("acct-fail")
    assert result is None, "failed refresh must return None, not raise"
 def test_refresh_without_credentials_returns_none():
    """_refresh_google_token must return None immediately when the OAuth client
    credentials are not configured — no DB query, no HTTP call."""
    with mock.patch("routes.email_helpers.os.environ.get", return_value=""):
        from routes.email_helpers import _refresh_google_token
        result = _refresh_google_token("acct-any")
    assert result is None
 # ── Password-account regression ───────────────────────────────────
 def test_imap_connect_uses_login_for_password_accounts():
    """Existing password-auth IMAP accounts must still call conn.login() and
    must NOT trigger the XOAUTH2 authenticate path."""
    from routes.email_helpers import _imap_connect
    mock_conn = mock.MagicMock()
    # _imap_connect calls _get_email_config internally — mock it to return our cfg.
    cfg = {
        "imap_host": "imap.gmail.com",
        "imap_port": 993,
        "imap_starttls": False,
        "imap_user": "me@gmail.com",
        "imap_password": "app-password-xyz",
        "oauth_provider": "",
        "account_id": "acct-pw",
    }
    with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
         mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
        _imap_connect("acct-pw", owner="alice")
    mock_conn.login.assert_called_once_with("me@gmail.com", "app-password-xyz")
    mock_conn.authenticate.assert_not_called()
 def test_imap_connect_uses_xoauth2_for_oauth_accounts():
    """OAuth accounts must call conn.authenticate('XOAUTH2', ...) and must NOT
    call conn.login() — which would fail without a password."""
    from routes.email_helpers import _imap_connect
    from src.secret_storage import encrypt as _enc
    mock_conn = mock.MagicMock()
    future_expiry = str(int(time.time()) + 7200)
    cfg = {
        "imap_host": "imap.gmail.com",
        "imap_port": 993,
        "imap_starttls": False,
        "imap_user": "me@nyu.edu",
        "imap_password": "",
        "oauth_provider": "google",
        "account_id": "acct-oauth",
        "oauth_access_token": _enc("ya29.live_token"),
        "oauth_token_expiry": future_expiry,
    }
    with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
         mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
        _imap_connect("acct-oauth", owner="alice")
    mock_conn.authenticate.assert_called_once()
    assert mock_conn.authenticate.call_args[0][0] == "XOAUTH2"
    mock_conn.login.assert_not_called()
@pytest.mark.asyncio
 async def test_account_list_response_does_not_expose_token_values():
    """The /accounts list route is the client-facing account inventory. It must
    expose `oauth_provider` (so the UI can show OAuth status) but never the
    access/refresh token values, encrypted or otherwise — only boolean
    has_*_password flags and the provider name."""
    from routes.email_routes import setup_email_routes
    from src.secret_storage import encrypt as _enc
    raw_access = "ya29.super_secret_access_token"
    raw_refresh = "1//super_secret_refresh_token"
    db, Factory = _make_db()
    _make_account(db, account_id="acct-list", owner="alice",
                  oauth_provider="google",
                  oauth_access_token=_enc(raw_access),
                  oauth_refresh_token=_enc(raw_refresh))
    db.close()
    router = setup_email_routes()
    list_accounts = None
    for route in router.routes:
        if route.path == "/api/email/accounts" and "GET" in getattr(route, "methods", set()):
            list_accounts = route.endpoint
            break
    assert list_accounts is not None, "accounts list route not found"
    with mock.patch("core.database.SessionLocal", Factory):
        result = await list_accounts(owner="alice")
    blob = json.dumps(result)
    assert raw_access not in blob, "raw access token must not appear in list response"
    assert raw_refresh not in blob, "raw refresh token must not appear in list response"
    assert _enc(raw_access) not in blob, "encrypted token must not be sent to the client either"
    acct = result["accounts"][0]
    assert acct["oauth_provider"] == "google"   # status is exposed
    assert "oauth_access_token" not in acct      # token value is not
    assert "oauth_refresh_token" not in acct
@@ -41,8 +41,10 @@ def _seed(tmp_path):
 def test_file_kept_when_commit_fails(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    SessionLocal = _seed(tmp_path)
    # GALLERY_IMAGE_DIR is an absolute path fixed at import, so a chdir can't
    # redirect the delete; point the resolver at the seeded tmp dir directly.
    monkeypatch.setattr(gallery_routes, "GALLERY_IMAGE_DIR", tmp_path / "data" / "generated_images")
    monkeypatch.setattr(gallery_routes, "get_current_user", lambda r: "alice")
    # A session whose commit always fails, to simulate a DB error mid-delete.
@@ -67,8 +69,8 @@ def test_file_kept_when_commit_fails(tmp_path, monkeypatch):
 def test_file_removed_on_successful_delete(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    SessionLocal = _seed(tmp_path)
    monkeypatch.setattr(gallery_routes, "GALLERY_IMAGE_DIR", tmp_path / "data" / "generated_images")
    monkeypatch.setattr(gallery_routes, "get_current_user", lambda r: "alice")
    monkeypatch.setattr(gallery_routes, "SessionLocal", SessionLocal)
@@ -2,7 +2,14 @@ import os
 from pathlib import Path
 import pytest
 from fastapi import FastAPI
 from fastapi import HTTPException
 from fastapi.testclient import TestClient
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker
 from sqlalchemy.pool import NullPool
 from core.database import Base, GalleryImage
 def _gallery_module():
@@ -53,6 +60,57 @@ def test_gallery_image_path_rejects_symlink_escape(tmp_path, monkeypatch):
    assert exc.value.status_code == 400
 def test_gallery_replace_rejects_symlink_escape(tmp_path, monkeypatch):
    gallery_routes = _gallery_module()
    image_dir = tmp_path / "generated_images"
    image_dir.mkdir()
    outside = tmp_path / "outside.png"
    outside.write_bytes(b"outside image root")
    link = image_dir / "escape.png"
    try:
        os.symlink(outside, link)
    except (AttributeError, NotImplementedError, OSError) as exc:
        pytest.skip(f"symlinks unavailable: {exc}")
    engine = create_engine(
        f"sqlite:///{tmp_path / 'gallery.db'}",
        connect_args={"check_same_thread": False},
        poolclass=NullPool,
    )
    Base.metadata.create_all(engine)
    SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False)
    db = SessionLocal()
    try:
        db.add(
            GalleryImage(
                id="img-1",
                filename="escape.png",
                prompt="escape",
                owner="alice",
                is_active=True,
            )
        )
        db.commit()
    finally:
        db.close()
    monkeypatch.setattr(gallery_routes, "GALLERY_IMAGE_DIR", image_dir)
    monkeypatch.setattr(gallery_routes, "SessionLocal", SessionLocal)
    monkeypatch.setattr(gallery_routes, "get_current_user", lambda request: "alice")
    app = FastAPI()
    app.include_router(gallery_routes.setup_gallery_routes())
    client = TestClient(app)
    response = client.post(
        "/api/gallery/img-1/replace",
        files={"image": ("replacement.png", b"replacement bytes", "image/png")},
    )
    assert response.status_code == 400
    assert outside.read_bytes() == b"outside image root"
 def test_gallery_file_operations_use_confining_resolver():
    source = Path("routes/gallery_routes.py").read_text(encoding="utf-8")
@@ -0,0 +1,59 @@
 from services.hwfit.fit import _lookup_apple_bandwidth, _lookup_bandwidth
 def test_m3_max_bandwidth_uses_gpu_cores():
    assert _lookup_bandwidth({"gpu_name": "Apple M3 Max", "gpu_cores": 30}) == 300
    assert _lookup_bandwidth({"gpu_name": "Apple M3 Max", "gpu_cores": 40}) == 400
 def test_m4_max_bandwidth_uses_gpu_cores():
    assert _lookup_bandwidth({"gpu_name": "Apple M4 Max", "gpu_cores": 32}) == 410
    assert _lookup_bandwidth({"gpu_name": "Apple M4 Max", "gpu_cores": 40}) == 546
 def test_m5_max_bandwidth_uses_gpu_cores():
    assert _lookup_bandwidth({"gpu_name": "Apple M5 Max", "gpu_cores": 32}) == 460
    assert _lookup_bandwidth({"gpu_name": "Apple M5 Max", "gpu_cores": 40}) == 614
 def test_apple_max_bandwidth_falls_back_conservatively_without_gpu_cores():
    assert _lookup_bandwidth({"gpu_name": "Apple M3 Max"}) == 300
    assert _lookup_bandwidth({"gpu_name": "Apple M4 Max"}) == 410
    assert _lookup_bandwidth({"gpu_name": "Apple M5 Max"}) == 460
 def test_fixed_apple_bandwidth_entries_include_updated_m5_values():
    assert _lookup_bandwidth({"gpu_name": "Apple M5 Pro"}) == 307
    assert _lookup_bandwidth({"gpu_name": "Apple M5"}) == 153
 def test_non_apple_gpu_does_not_match_apple_bandwidth():
    """NVIDIA Quadro M4 000 should NOT match Apple bandwidth lookup."""
    assert _lookup_bandwidth({"gpu_name": "NVIDIA Quadro M4 000"}) is None
    assert _lookup_bandwidth({"gpu_name": "NVIDIA Quadro M3 000"}) is None
    assert _lookup_bandwidth({"gpu_name": "NVIDIA Quadro M5 000"}) is None
 def test_non_apple_gpu_with_cores_does_not_match():
    """A non-Apple GPU that happens to carry a gpu_cores count must not be
    matched by the APPLE bandwidth path. This asserts the Apple-specific
    matcher directly: _lookup_bandwidth would (correctly) return these cards'
    real bandwidth from the general GPU table (e.g. the RTX 4090's 1008 GB/s),
    which is a different code path and not what this guard is about.
    """
    assert _lookup_apple_bandwidth({"gpu_name": "NVIDIA GeForce RTX 4090", "gpu_cores": 128}) is None
    assert _lookup_apple_bandwidth({"gpu_name": "AMD Radeon RX 9070 XT", "gpu_cores": 64}) is None
 def test_apple_string_input_resolves_conservative_tier():
    """Bare-string callers must still get Apple bandwidth. #2564 moved the
    Apple tiers out of the generic GPU table into the dict-only Apple helper,
    so _lookup_bandwidth("Apple M3 Max") (no gpu_cores) regressed to None;
    string inputs now route through the Apple helper and get the conservative
    (lowest) tier for the model."""
    assert _lookup_bandwidth("Apple M3 Max") == 300
    assert _lookup_bandwidth("Apple M4 Max") == 410
    assert _lookup_bandwidth("Apple M5 Max") == 460
    # Non-Apple strings still fall through to the generic table.
    assert _lookup_bandwidth("NVIDIA GeForce RTX 4090") == 1008
    assert _lookup_bandwidth("Totally Unknown GPU") is None
@@ -4,6 +4,8 @@ Covers the Metal-specific behavior added for Apple Silicon and locks in the
 guarantee that non-macOS (Linux/Windows) detection is unchanged.
 """
 import json
 from services.hwfit import hardware
 from services.hwfit.fit import rank_models
 from services.hwfit.models import get_models
@@ -22,7 +24,7 @@ def _metal_system(ram_gb=16.0, vram_gb=10.7):
    }
-def _fake_sysctl(brand="Apple M2 Pro", memsize_gb=32, wired_mb=None):
+def _fake_sysctl(brand="Apple M2 Pro", memsize_gb=32, wired_mb=None, display_json=None, display_text=None):
    def run(cmd):
        joined = " ".join(cmd)
        if "machdep.cpu.brand_string" in joined:
@@ -31,6 +33,12 @@ def _fake_sysctl(brand="Apple M2 Pro", memsize_gb=32, wired_mb=None):
            return str(int(memsize_gb * 1024**3))
        if "iogpu.wired_limit_mb" in joined:
            return str(wired_mb) if wired_mb is not None else None
        if "system_profiler SPDisplaysDataType -json" in joined:
            if isinstance(display_json, (dict, list)):
                return json.dumps(display_json)
            return display_json
        if "system_profiler SPDisplaysDataType" in joined:
            return display_text
        return None
    return run
@@ -98,16 +106,47 @@ def test_apple_silicon_detected_as_metal(monkeypatch):
    monkeypatch.setattr(hardware, "_remote_host", None)
    monkeypatch.setattr(hardware.platform, "system", lambda: "Darwin")
    monkeypatch.setattr(hardware.platform, "machine", lambda: "arm64")
-    monkeypatch.setattr(hardware, "_run", _fake_sysctl(memsize_gb=32))
+    monkeypatch.setattr(hardware, "_run", _fake_sysctl(
        memsize_gb=32,
        display_json={"SPDisplaysDataType": [{"sppci_model": "Apple M2 Pro", "sppci_cores": "19"}]},
    ))
    info = hardware._detect_apple_silicon()
    assert info is not None
    assert info["backend"] == "metal"
    assert info["gpu_name"] == "Apple M2 Pro"
    assert info["unified_memory"] is True
    assert info["gpu_cores"] == 19
    assert info["gpu_vram_gb"] == 24.0  # 32GB * 0.75
 def test_apple_silicon_gpu_cores_fall_back_to_plain_text(monkeypatch):
    monkeypatch.setattr(hardware, "_remote_host", None)
    monkeypatch.setattr(hardware.platform, "system", lambda: "Darwin")
    monkeypatch.setattr(hardware.platform, "machine", lambda: "arm64")
    monkeypatch.setattr(hardware, "_run", _fake_sysctl(
        brand="Apple M4 Max",
        memsize_gb=64,
        display_json="{not-json",
        display_text="Graphics/Displays:\n\nApple M4 Max:\n  Total Number of Cores: 32\n",
    ))
    info = hardware._detect_apple_silicon()
    assert info is not None
    assert info["gpu_cores"] == 32
 def test_apple_silicon_gpu_cores_are_optional(monkeypatch):
    monkeypatch.setattr(hardware, "_remote_host", None)
    monkeypatch.setattr(hardware.platform, "system", lambda: "Darwin")
    monkeypatch.setattr(hardware.platform, "machine", lambda: "arm64")
    monkeypatch.setattr(hardware, "_run", _fake_sysctl(memsize_gb=32))
    info = hardware._detect_apple_silicon()
    assert info is not None
    assert "gpu_cores" not in info
 def test_apple_silicon_skipped_on_linux(monkeypatch):
    """Guarantee Linux detection is untouched: the Metal probe bails immediately."""
    monkeypatch.setattr(hardware, "_remote_host", None)
@@ -132,7 +171,7 @@ def test_detect_system_propagates_unified_memory(monkeypatch):
    monkeypatch.setattr(hardware, "_detect_apple_silicon", lambda: {
        "gpu_name": "Apple M4", "gpu_vram_gb": 10.7, "gpu_count": 1,
        "gpus": [], "gpu_groups": [], "homogeneous": True,
-        "backend": "metal", "unified_memory": True,
+        "backend": "metal", "unified_memory": True, "gpu_cores": 10,
    })
    monkeypatch.setattr(hardware, "_get_ram_gb", lambda: 16.0)
    monkeypatch.setattr(hardware, "_get_available_ram_gb", lambda: 11.0)
@@ -142,3 +181,4 @@ def test_detect_system_propagates_unified_memory(monkeypatch):
    s = hardware.detect_system(fresh=True)
    assert s["backend"] == "metal"
    assert s.get("unified_memory") is True
    assert s["gpu_cores"] == 10
@@ -79,7 +79,7 @@ def _build_context_harness(monkeypatch, chat_helpers, history):
    monkeypatch.setattr(chat_helpers, "extract_preset", fake_extract_preset)
    monkeypatch.setattr(chat_helpers, "add_user_message", fake_add_user_message)
    monkeypatch.setattr(chat_helpers, "load_prefs_for_user", lambda user: {})
-    monkeypatch.setattr(chat_helpers, "get_current_user", lambda request: "tester")
+    monkeypatch.setattr(chat_helpers, "effective_user", lambda request: "tester")
    monkeypatch.setattr(chat_helpers, "normalize_model_id", lambda endpoint_url, model, **kwargs: None)
    monkeypatch.setattr(chat_helpers, "maybe_compact", fake_maybe_compact)
    monkeypatch.setattr(chat_helpers, "trim_for_context", lambda messages, context_length: messages)
@@ -1286,6 +1286,14 @@ class _ImmediateThread:
        self.target()
 class _NoopThread:
    def __init__(self, target, daemon=None):
        self.target = target
    def start(self):
        return None
 def _wait_for(predicate, timeout=2.0):
    deadline = time.time() + timeout
    while time.time() < deadline:
@@ -1313,6 +1321,7 @@ def _route_ep(
    pinned_models=None,
    refresh_mode="auto",
    refresh_timeout=None,
    owner=None,
 ):
    return SimpleNamespace(
        id=id,
@@ -1329,7 +1338,7 @@ def _route_ep(
        model_refresh_interval=None,
        model_refresh_timeout=refresh_timeout,
        supports_tools=None,
-        owner=None,
+        owner=owner,
        created_at=None,
        updated_at=None,
    )
@@ -1342,6 +1351,72 @@ def _route_request():
    )
 def test_api_models_rejects_api_token_without_chat_scope(monkeypatch):
    router = model_routes.setup_model_routes(model_discovery=None)
    def fail_session():
        raise AssertionError("model DB should not be queried without chat scope")
    monkeypatch.setattr(model_routes, "SessionLocal", fail_session)
    request = SimpleNamespace(
        state=SimpleNamespace(
            current_user="api",
            api_token=True,
            api_token_owner="alice",
            api_token_scopes=["documents:read"],
        ),
        app=SimpleNamespace(
            state=SimpleNamespace(
                auth_manager=SimpleNamespace(is_configured=True, is_admin=lambda user: False),
            ),
        ),
    )
    with pytest.raises(HTTPException) as exc:
        _route_endpoint(router, "/api/models")(request)
    assert exc.value.status_code == 403
    assert "chat" in str(exc.value.detail)
 def test_api_models_scopes_api_token_to_token_owner(monkeypatch):
    rows = [
        _route_ep("alice", "http://alice.example/v1", cached_models=["alice-model"], owner="alice"),
        _route_ep("shared", "http://shared.example/v1", cached_models=["shared-model"], owner=None),
        _route_ep("bob", "http://bob.example/v1", cached_models=["bob-model"], owner="bob"),
    ]
    db = _RouteDb(rows)
    router = model_routes.setup_model_routes(model_discovery=None)
    admin_checks = []
    monkeypatch.setattr(model_routes, "ModelEndpoint", _RouteModelEndpoint)
    monkeypatch.setattr(model_routes, "SessionLocal", lambda: db)
    monkeypatch.setattr(threading, "Thread", _NoopThread)
    request = SimpleNamespace(
        state=SimpleNamespace(
            current_user="api",
            api_token=True,
            api_token_owner="alice",
            api_token_scopes=["chat"],
        ),
        app=SimpleNamespace(
            state=SimpleNamespace(
                auth_manager=SimpleNamespace(
                    is_configured=True,
                    is_admin=lambda user: admin_checks.append(user) or False,
                ),
            ),
        ),
    )
    result = _route_endpoint(router, "/api/models")(request)
    assert [item["endpoint_name"] for item in result["items"]] == ["alice", "shared"]
    assert admin_checks == ["alice"]
 def test_api_models_returns_cached_proxy_models_without_refresh_probe(monkeypatch):
    row = _route_ep(
        "proxy",
@@ -0,0 +1,188 @@
 """Owner-scoped note routes must fail closed when the request has no identity.
 The notes CRUD routes resolved the acting user with bare get_current_user().
 A request that reached them carrying no identity (auth-middleware regression,
 SSRF from a sibling service) therefore came through as user=None — and the
 queries treat None as the single-user mode, i.e. blanket access to every
 account's notes: list everything, read/update/delete/pin/archive any row,
 reorder globally.
 require_user() already encodes the correct policy — 401 when auth is
 configured, while the documented anonymous modes (AUTH_ENABLED=false,
 LOCALHOST_BYPASS on loopback, unconfigured first-run) still pass — and
 fire-reminder in the same file already used it. The CRUD routes now resolve
 the owner through it too.
 Test transport note: these drive the ASGI app through ``httpx.ASGITransport``
 + ``httpx.AsyncClient`` rather than ``starlette.testclient.TestClient``.
 TestClient runs the app inside a background event-loop thread spun up by
 ``anyio.from_thread.start_blocking_portal`` and then dispatches each sync
 endpoint onto *another* worker thread; on some anyio/httpx/platform
 combinations that two-thread handshake deadlocks and ``TestClient(app).get(...)``
 simply hangs. ASGITransport runs the whole request on the test's own event
 loop — no portal thread, no BaseHTTPMiddleware — so the suite is portable.
 Identity is injected by a pure-ASGI shim that writes the same
 ``request.state`` fields the real auth middleware sets.
 """
 import uuid
 from types import SimpleNamespace
 import httpx
 import pytest
 from fastapi import FastAPI
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker
 from sqlalchemy.pool import NullPool
 import core.database as cdb
 from core.database import Note
 import routes.note_routes as nr
 # A deliberately NON-loopback peer. require_user has loopback fall-throughs
 # (unconfigured first-run, LOCALHOST_BYPASS); pinning a public-looking client
 # keeps every assertion below about the *configured-auth* path and not an
 # accidental loopback bypass — the same reason the old fixture leaned on
 # TestClient's non-loopback "testclient" host.
 _PEER = ("203.0.113.7", 54321)
 class _Identity:
    """Pure-ASGI shim mirroring what the auth middleware writes onto
    request.state. Pure-ASGI on purpose — it stays off Starlette's
    BaseHTTPMiddleware + sync-TestClient path, the source of the
    ``TestClient(app).get(...)`` hang. No x-test-user header => no identity,
    the exact state an auth-middleware regression would produce."""
    def __init__(self, app):
        self.app = app
    async def __call__(self, scope, receive, send):
        if scope["type"] == "http":
            headers = dict(scope.get("headers") or [])
            state = scope.setdefault("state", {})
            user = headers.get(b"x-test-user")
            if user:
                state["current_user"] = user.decode()
            if headers.get(b"x-test-api-token"):
                state["current_user"] = "api"
                state["api_token"] = True
        await self.app(scope, receive, send)
 def _temp_db(tmp_path):
    """Note routes over a fresh temp DB; returns the session factory."""
    engine = create_engine(
        f"sqlite:///{tmp_path / 'notes.db'}",
        connect_args={"check_same_thread": False},
        poolclass=NullPool,
    )
    cdb.Base.metadata.create_all(engine)
    return sessionmaker(bind=engine)
 def _build_app(factory, *, configured=True):
    app = FastAPI()
    app.state.auth_manager = SimpleNamespace(is_configured=configured)
    app.include_router(nr.setup_note_routes())
    return _Identity(app)
 def _client(app):
    """AsyncClient over the ASGI app with a non-loopback peer. Caller drives
    it inside ``async with``."""
    transport = httpx.ASGITransport(app=app, client=_PEER)
    return httpx.AsyncClient(transport=transport, base_url="http://notes.test")
@pytest.fixture
 def env(monkeypatch, tmp_path):
    """Configured-auth world: AUTH_ENABLED=true, auth_manager.is_configured,
    no LOCALHOST_BYPASS. Identity comes only from the x-test-user header
    (mirroring the auth middleware); no header => no identity, the exact state
    an auth-middleware regression leaves behind. Seeds one note each for alice
    and bob. Returns (app, factory)."""
    factory = _temp_db(tmp_path)
    monkeypatch.setattr(nr, "SessionLocal", factory)
    monkeypatch.setenv("AUTH_ENABLED", "true")
    monkeypatch.delenv("LOCALHOST_BYPASS", raising=False)
    app = _build_app(factory)
    db = factory()
    db.add(Note(id="note-alice", owner="alice", title="a", content="x",
                items='[{"text": "t", "done": false}]'))
    db.add(Note(id="note-bob", owner="bob", title="b", content="y"))
    db.commit()
    db.close()
    return app, factory
 async def test_no_identity_fails_closed_on_every_owner_scoped_route(env):
    app, _ = env
    async with _client(app) as c:
        assert (await c.get("/api/notes")).status_code == 401
        assert (await c.get("/api/notes/note-alice")).status_code == 401
        assert (await c.put("/api/notes/note-alice", json={"title": "pwn"})).status_code == 401
        assert (await c.delete("/api/notes/note-alice")).status_code == 401
        assert (await c.post("/api/notes/note-alice/pin")).status_code == 401
        assert (await c.post("/api/notes/note-alice/archive")).status_code == 401
        assert (await c.post("/api/notes/note-alice/items/0/toggle")).status_code == 401
        assert (await c.post("/api/notes/reorder", json={"ids": ["note-bob", "note-alice"]})).status_code == 401
        assert (await c.post("/api/notes", json={"title": "ghost"})).status_code == 401
 async def test_no_identity_did_not_mutate_anything(env):
    app, factory = env
    async with _client(app) as c:
        await c.put("/api/notes/note-alice", json={"title": "pwn"})
        await c.post("/api/notes/note-alice/pin")
        await c.delete("/api/notes/note-bob")
    db = factory()
    rows = {n.id: n for n in db.query(Note).all()}
    db.close()
    assert set(rows) == {"note-alice", "note-bob"}
    assert rows["note-alice"].title == "a"
    assert not rows["note-alice"].pinned
 async def test_authenticated_user_still_scoped_to_own_notes(env):
    app, _ = env
    alice = {"x-test-user": "alice"}
    async with _client(app) as c:
        listed = (await c.get("/api/notes", headers=alice)).json()["notes"]
        assert [n["id"] for n in listed] == ["note-alice"]
        assert (await c.get("/api/notes/note-alice", headers=alice)).status_code == 200
        # Someone else's note stays a 404 (don't reveal it exists).
        assert (await c.get("/api/notes/note-bob", headers=alice)).status_code == 404
        assert (await c.put("/api/notes/note-alice", json={"title": "mine"}, headers=alice)).status_code == 200
 async def test_api_token_pseudo_user_is_rejected(env):
    """Bearer tokens must use the scope-aware API routes (require_user's
    existing contract), not slip into cookie-session routes as user 'api'."""
    app, _ = env
    async with _client(app) as c:
        r = await c.get("/api/notes", headers={"x-test-api-token": "1"})
    assert r.status_code == 403
 async def test_auth_disabled_keeps_single_user_mode_working(monkeypatch, tmp_path):
    """AUTH_ENABLED=false is the operator's explicit anonymous mode: no
    identity must still mean full single-user access (issue #622 contract),
    even with a stale configured auth.json on disk."""
    factory = _temp_db(tmp_path)
    monkeypatch.setattr(nr, "SessionLocal", factory)
    monkeypatch.setenv("AUTH_ENABLED", "false")
    app = _build_app(factory)
    db = factory()
    db.add(Note(id="n1", owner=None, title="solo", content="x"))
    db.commit()
    db.close()
    async with _client(app) as c:
        assert [n["id"] for n in (await c.get("/api/notes")).json()["notes"]] == ["n1"]
        assert (await c.put("/api/notes/n1", json={"title": "still mine"})).status_code == 200
        assert (await c.post("/api/notes/n1/pin")).status_code == 200
@@ -0,0 +1,74 @@
 import asyncio
 import os
 from pathlib import Path
 from routes import personal_routes
 class _FakePersonalDocs:
    def __init__(self):
        self.excluded = []
    def exclude_file(self, filepath):
        self.excluded.append(filepath)
 class _FakeRAG:
    def __init__(self):
        self.deleted_sources = []
    def delete_by_source(self, filepath):
        self.deleted_sources.append(filepath)
        return 1
 def _delete_endpoint(personal_docs):
    router = personal_routes.setup_personal_routes(personal_docs, None, True)
    for route in router.routes:
        if getattr(route, "path", "") == "/api/personal/file" and "DELETE" in getattr(route, "methods", set()):
            return route.endpoint
    raise AssertionError("DELETE /api/personal/file endpoint not found")
 def test_delete_file_refuses_symlink_directory_escape(tmp_path, monkeypatch):
    uploads = tmp_path / "uploads"
    uploads.mkdir()
    outside = tmp_path / "outside"
    outside.mkdir()
    victim = outside / "victim.txt"
    victim.write_text("keep me", encoding="utf-8")
    os.symlink(outside, uploads / "linked")
    docs = _FakePersonalDocs()
    rag = _FakeRAG()
    monkeypatch.setattr(personal_routes, "UPLOADS_DIR", str(uploads))
    monkeypatch.setattr(personal_routes, "get_rag_manager", lambda: rag)
    filepath = str(uploads / "linked" / "victim.txt")
    result = asyncio.run(_delete_endpoint(docs)(filepath=filepath, owner="alice", _admin=None))
    assert result["deleted_from_disk"] is False
    assert victim.read_text(encoding="utf-8") == "keep me"
    assert docs.excluded == [filepath]
    assert rag.deleted_sources == [filepath]
 def test_delete_file_removes_regular_file_inside_upload_root(tmp_path, monkeypatch):
    uploads = tmp_path / "uploads"
    uploads.mkdir()
    uploaded_file = uploads / "alice" / "notes.txt"
    uploaded_file.parent.mkdir()
    uploaded_file.write_text("delete me", encoding="utf-8")
    docs = _FakePersonalDocs()
    rag = _FakeRAG()
    monkeypatch.setattr(personal_routes, "UPLOADS_DIR", str(uploads))
    monkeypatch.setattr(personal_routes, "get_rag_manager", lambda: rag)
    filepath = str(uploaded_file)
    result = asyncio.run(_delete_endpoint(docs)(filepath=filepath, owner="alice", _admin=None))
    assert result["deleted_from_disk"] is True
    assert not uploaded_file.exists()
    assert docs.excluded == [filepath]
    assert rag.deleted_sources == [filepath]
@@ -107,6 +107,7 @@ class TestBuildersRejectLookalikeHosts:
        assert build_chat_url("https://notanthropic.com") == "https://notanthropic.com/chat/completions"
    def test_lookalike_anthropic_models_is_openai(self):
        assert llm_core._detect_provider("https://anthropic.com.evil.com") == "openai"
        assert build_models_url("https://anthropic.com.evil.com") == "https://anthropic.com.evil.com/models"
    def test_anthropic_domain_in_path_is_openai(self):
@@ -119,6 +120,7 @@ class TestBuildersRejectLookalikeHosts:
        assert build_chat_url("https://notollama.com") == "https://notollama.com/chat/completions"
    def test_lookalike_ollama_models_is_openai(self):
        assert llm_core._detect_provider("https://notollama.com") == "openai"
        assert build_models_url("https://notollama.com") == "https://notollama.com/models"
@@ -1,16 +1,18 @@
-"""Regression guard for issue #1390 — the README banner / ASCII art was not in a
+"""Regression guard for the README title presentation.
 fenced code block, so GitHub's markdown collapsed its leading whitespace and the
 box-drawing rules, rendering it misaligned instead of monospace-as-typed.
-This pins that the decorative banner stays inside a ``` code fence.
+Originally (#1390) the README opened with an ASCII-art banner that had to live
 inside a ``` code fence, otherwise GitHub's markdown collapsed its leading
 whitespace and box-drawing rules and rendered it misaligned. The README refresh
 (#4306) dropped that banner in favour of a centered wordmark image, so the guard
 now pins the wordmark identity instead, while still catching the original failure
 mode if an un-fenced ASCII banner is ever reintroduced.
 """
 from pathlib import Path
 README = Path(__file__).resolve().parent.parent / "README.md"
-# Distinctive bits of the banner (box-drawing rule + the kaomoji version line).
+# Box-drawing rule from the legacy ASCII banner (the #1390 failure mode).
 _RULE = "─" * 10
 _BANNER_LINE = "Odysseus vers. 1.0"
 def _fenced_segments(text: str):
@@ -20,15 +22,18 @@ def _fenced_segments(text: str):
    return parts[1::2]
-def test_readme_banner_is_inside_a_code_fence():
+def test_readme_opens_with_wordmark_title():
    # The README must still open with a recognizable Odysseus title: now the
    # centered wordmark image rather than an H1 / ASCII banner.
    head = "\n".join(README.read_text(encoding="utf-8").splitlines()[:15])
    assert 'alt="Odysseus"' in head, "README must open with the Odysseus wordmark image"
 def test_reintroduced_ascii_banner_stays_fenced():
    # Defensive: if a box-drawing banner is ever added back, it must be fenced so
    # GitHub renders it monospace-as-typed (the original #1390 regression).
    text = README.read_text(encoding="utf-8")
-    assert _BANNER_LINE in text, "banner line missing from README"
+    if _RULE not in text:
        return
    inside = "\n".join(_fenced_segments(text))
-    assert _BANNER_LINE in inside, "banner version line must be inside a ``` code fence"
+    assert _RULE in inside, "ASCII banner rule must be inside a ``` code fence"
    assert _RULE in inside, "banner rule line must be inside a ``` code fence"
 def test_readme_title_stays_a_heading():
    # The H1 must remain a real heading, not get swallowed into the fence.
    first = README.read_text(encoding="utf-8").splitlines()[0]
    assert first.strip() == "# Odysseus"
@@ -385,7 +385,7 @@ async def test_build_chat_context_incognito_does_not_duplicate_current_user_mess
    monkeypatch.setattr(chat_helpers, "extract_preset", fake_extract_preset)
    monkeypatch.setattr(chat_helpers, "add_user_message", fake_add_user_message)
    monkeypatch.setattr(chat_helpers, "load_prefs_for_user", lambda user: {})
-    monkeypatch.setattr(chat_helpers, "get_current_user", lambda request: "tester")
+    monkeypatch.setattr(chat_helpers, "effective_user", lambda request: "tester")
    monkeypatch.setattr(chat_helpers, "normalize_model_id", lambda endpoint_url, model, **kwargs: None)
    monkeypatch.setattr(chat_helpers, "maybe_compact", fake_maybe_compact)
    monkeypatch.setattr(chat_helpers, "trim_for_context", lambda messages, context_length: messages)
@@ -0,0 +1,50 @@
 import os
 import sys
 from unittest import mock
 import pytest
 from src.runtime_paths import get_app_root, get_default_data_dir
 def test_get_app_root_normal_run():
    """Verify that get_app_root returns the repository root parent of src/ when not frozen."""
    with mock.patch.object(sys, "frozen", False, create=True):
        app_root = get_app_root()
        # Verify it is a valid directory path and matches expected parent structure
        assert os.path.isdir(app_root)
        assert os.path.exists(os.path.join(app_root, "src"))
 def test_get_app_root_frozen_with_meipass():
    """Verify that get_app_root returns the sys._MEIPASS directory when frozen by PyInstaller."""
    mock_meipass = os.path.abspath("mock_meipass_dir")
    with mock.patch.object(sys, "frozen", True, create=True), \
         mock.patch.object(sys, "_MEIPASS", mock_meipass, create=True):
        app_root = get_app_root()
        assert app_root == mock_meipass
 def test_get_app_root_frozen_without_meipass():
    """Verify that get_app_root falls back to the sys.executable parent directory when frozen but _MEIPASS is absent."""
    mock_exe_path = os.path.join(os.path.abspath("mock_exe_dir"), "Odysseus.exe")
    with mock.patch.object(sys, "frozen", True, create=True), \
         mock.patch.object(sys, "executable", mock_exe_path, create=True):
        # Remove sys._MEIPASS if it exists in the test process environment
        if hasattr(sys, "_MEIPASS"):
            delattr(sys, "_MEIPASS")
        app_root = get_app_root()
        assert app_root == os.path.abspath("mock_exe_dir")
 def test_get_default_data_dir_normal():
    """Verify that get_default_data_dir resolves to get_app_root() / 'data' when not frozen."""
    with mock.patch.object(sys, "frozen", False, create=True):
        res = get_default_data_dir()
        assert res == os.path.join(get_app_root(), "data")
 def test_get_default_data_dir_frozen():
    """Verify that get_default_data_dir resolves to a persistent user path under ~ when frozen."""
    with mock.patch.object(sys, "frozen", True, create=True):
        res = get_default_data_dir()
        expected = os.path.join(os.path.expanduser("~"), ".odysseus", "data")
        assert res == expected
@@ -58,7 +58,7 @@ def test_content_fetcher_extracts_og_image_and_body_fallback(module, tmp_path, m
    monkeypatch.setattr(module, "CONTENT_CACHE_DIR", tmp_path)
    module.content_cache_index.clear()
-    monkeypatch.setattr(module, "_get_public_url", lambda url, headers, timeout: _FakeResponse(html))
+    monkeypatch.setattr(module, "_get_public_url", lambda url, headers, timeout, **kwargs: _FakeResponse(html))
    result = module.fetch_webpage_content("https://example.com/parity-test")
@@ -82,7 +82,7 @@ def test_fetch_webpage_content_returns_empty_result_on_http_status_error(status_
    monkeypatch.setattr(
        service_content,
        "_get_public_url",
-        lambda url, headers, timeout: _FakeErrorResponse(status_code),
+        lambda url, headers, timeout, **kwargs: _FakeErrorResponse(status_code),
    )
    result = service_content.fetch_webpage_content(f"https://example.com/status-{status_code}")
@@ -119,7 +119,7 @@ def test_fetch_webpage_content_429_takes_distinct_rate_limit_path(tmp_path, monk
    monkeypatch.setattr(
        service_content,
        "_get_public_url",
-        lambda url, headers, timeout: _FakeRateLimitResponse(),
+        lambda url, headers, timeout, **kwargs: _FakeRateLimitResponse(),
    )
    result = service_content.fetch_webpage_content("https://example.com/rate-limited")
@@ -121,9 +121,12 @@ def test_docker_compose_binds_web_ui_to_loopback_by_default():
 def test_readme_native_quickstart_uses_loopback():
-    readme = Path("README.md").read_text(encoding="utf-8")
+    # The README refresh (#4306) moved the native quickstart into docs/setup.md,
-    assert "python -m uvicorn app:app --host 127.0.0.1 --port 7000" in readme
+    # so accept the loopback guidance from either the README or the setup guide.
-    assert "0.0.0.0` only when you intentionally want" in readme
+    docs = Path("README.md").read_text(encoding="utf-8")
    docs += "\n" + Path("docs/setup.md").read_text(encoding="utf-8")
    assert "python -m uvicorn app:app --host 127.0.0.1 --port 7000" in docs
    assert "0.0.0.0` only when you intentionally want" in docs
 def test_ollama_cookbook_runner_does_not_force_public_bind():
@@ -901,7 +904,13 @@ def test_web_fetch_guard_blocks_redirect_into_private(monkeypatch):
        url = "http://public.example/start"
        headers = {"location": "http://169.254.169.254/latest/meta-data/"}
-    monkeypatch.setattr(httpx, "get", lambda url, **kwargs: _Resp())
+    from contextlib import contextmanager
    @contextmanager
    def _fake_stream(method, url, **kwargs):
        yield _Resp()
    monkeypatch.setattr(httpx, "stream", _fake_stream)
    with _pytest.raises(httpx.RequestError) as exc:
        content._get_public_url("http://public.example/start", headers={}, timeout=5)
@@ -52,6 +52,6 @@ def test_chat_endpoint_recovery_paths_are_owner_scoped():
    assert "def _clear_orphaned_session_endpoint(sess, owner:" in chat_routes
    assert "def _recover_empty_session_model(sess, session_id: str, owner:" in chat_routes
    assert "q = owner_filter(q, ModelEndpoint, owner)" in chat_routes
-    assert "resolve_session_auth(sess, session, owner=get_current_user(request))" in chat_routes
+    assert "resolve_session_auth(sess, session, owner=effective_user(request))" in chat_routes
    assert "def resolve_session_auth(sess, session_id: str, owner:" in chat_helpers
    assert "update_q = update_q.filter(DBSession.owner == owner)" in chat_helpers
@@ -35,7 +35,7 @@ def _patch_fetch(monkeypatch, text, content_type):
    monkeypatch.setattr(
        content_mod,
        "_get_public_url",
-        lambda url, headers=None, timeout=5: _FakeResponse(text, content_type),
+        lambda url, headers=None, timeout=5, **kwargs: _FakeResponse(text, content_type),
    )
@@ -0,0 +1,206 @@
 """web_fetch download budgets (#3812).
 MAX_OUTPUT_CHARS only trims what the agent sees; these caps bound what the
 server downloads, parses, and caches. Soft cap by default with a truncation
 notice, per-call override clamped to the hard cap, and a pre-buffer refusal
 when Content-Length already exceeds the hard ceiling.
 """
 import json
 from contextlib import contextmanager
 import pytest
 from src.constants import WEB_FETCH_SOFT_MAX_BYTES, WEB_FETCH_HARD_MAX_BYTES
 from services.search import content as content_mod
 class _FakeStream:
    """Stands in for the httpx.stream(...) context manager."""
    def __init__(self, body: bytes, content_type="text/plain", content_length=None,
                 status_code=200, chunk=8192):
        self._body = body
        self._chunk = chunk
        self.status_code = status_code
        self.encoding = "utf-8"
        self.url = "https://example.com/x"
        self.headers = {"Content-Type": content_type}
        if content_length is not None:
            self.headers["content-length"] = str(content_length)
        self.body_reads = 0
    def iter_bytes(self):
        for i in range(0, len(self._body), self._chunk):
            self.body_reads += 1
            yield self._body[i:i + self._chunk]
@pytest.fixture
 def no_cache(monkeypatch, tmp_path):
    monkeypatch.setattr(content_mod, "CONTENT_CACHE_DIR", tmp_path)
    monkeypatch.setattr(content_mod, "_cache_result", lambda *a, **k: None)
    monkeypatch.setattr(content_mod, "_public_http_url", lambda u: True)
 def _patch_stream(monkeypatch, fake):
    @contextmanager
    def fake_stream(method, url, **kwargs):
        yield fake
    monkeypatch.setattr(content_mod.httpx, "stream", fake_stream)
    return fake
 def test_body_under_cap_is_untouched(monkeypatch, no_cache):
    _patch_stream(monkeypatch, _FakeStream(b"hello world"))
    r = content_mod.fetch_webpage_content("https://example.com/a.txt")
    assert r["success"] is True
    assert r["content"] == "hello world"
    assert r["truncated"] is False
    assert r["fetched_bytes"] == len(b"hello world")
 def test_body_over_soft_cap_truncates_with_flags(monkeypatch, no_cache):
    body = b"x" * (WEB_FETCH_SOFT_MAX_BYTES + 50_000)
    _patch_stream(monkeypatch, _FakeStream(body, content_length=len(body)))
    r = content_mod.fetch_webpage_content("https://example.com/big.txt")
    assert r["truncated"] is True
    assert r["fetched_bytes"] == WEB_FETCH_SOFT_MAX_BYTES
    assert r["total_bytes"] == len(body)
    assert len(r["content"]) == WEB_FETCH_SOFT_MAX_BYTES
 def test_max_bytes_override_raises_budget(monkeypatch, no_cache):
    body = b"y" * (WEB_FETCH_SOFT_MAX_BYTES + 50_000)
    _patch_stream(monkeypatch, _FakeStream(body))
    r = content_mod.fetch_webpage_content(
        "https://example.com/big.txt", max_bytes=len(body) + 1
    )
    assert r["truncated"] is False
    assert r["fetched_bytes"] == len(body)
 def test_override_is_clamped_to_hard_cap(monkeypatch, no_cache):
    # Ask for more than the ceiling; the effective budget must be the ceiling.
    fake = _patch_stream(monkeypatch, _FakeStream(b"z" * 10, chunk=4))
    r = content_mod.fetch_webpage_content(
        "https://example.com/a.txt", max_bytes=WEB_FETCH_HARD_MAX_BYTES * 10
    )
    assert r["success"] is True
    # The clamp itself: effective cap recorded in the cache key path is the
    # hard cap, and a declared body over the ceiling is refused regardless.
    big = _FakeStream(b"", content_length=WEB_FETCH_HARD_MAX_BYTES + 1)
    _patch_stream(monkeypatch, big)
    r = content_mod.fetch_webpage_content(
        "https://example.com/huge.bin", max_bytes=WEB_FETCH_HARD_MAX_BYTES * 10
    )
    assert r["success"] is False
    assert "TooLarge" in r["error"]
    assert big.body_reads == 0  # refused before buffering
 def test_declared_over_hard_cap_refused_before_buffering(monkeypatch, no_cache):
    fake = _FakeStream(b"irrelevant", content_length=WEB_FETCH_HARD_MAX_BYTES + 1)
    _patch_stream(monkeypatch, fake)
    r = content_mod.fetch_webpage_content("https://example.com/huge.iso")
    assert r["success"] is False
    assert "TooLarge" in r["error"]
    assert fake.body_reads == 0
 def test_truncated_pdf_is_an_error_not_garbage(monkeypatch, no_cache):
    body = b"%PDF-1.4 " + b"p" * (WEB_FETCH_SOFT_MAX_BYTES + 10)
    _patch_stream(monkeypatch, _FakeStream(body, content_type="application/pdf"))
    r = content_mod.fetch_webpage_content("https://example.com/big.pdf")
    assert r["success"] is False
    assert "TooLarge" in r["error"]
 def test_fetch_requests_identity_encoding(monkeypatch, no_cache):
    # Compressed responses can decode to far more than Content-Length, so the
    # streamed cap and the hard-cap preflight are only honest when we refuse
    # transfer compression. Pin that the fetch advertises identity, not gzip.
    seen = {}
    @contextmanager
    def fake_stream(method, url, **kwargs):
        seen["headers"] = kwargs.get("headers") or {}
        yield _FakeStream(b"hello")
    monkeypatch.setattr(content_mod.httpx, "stream", fake_stream)
    content_mod.fetch_webpage_content("https://example.com/a.txt")
    assert seen["headers"].get("Accept-Encoding") == "identity"
 def test_rejects_compressed_response_that_ignored_identity(monkeypatch, no_cache):
    # We request Accept-Encoding: identity, but a server can ignore it and send
    # gzip anyway. httpx would decode it, so a tiny compressed body could balloon
    # past the cap in one decoded chunk. Refuse before reading the body.
    fake = _FakeStream(b"x" * 5000, content_length=40)
    fake.headers["content-encoding"] = "gzip"
    _patch_stream(monkeypatch, fake)
    r = content_mod.fetch_webpage_content("https://example.com/a.txt")
    assert r["success"] is False
    assert "Content-Encoding" in r["error"] or "compressed" in r["error"]
    assert fake.body_reads == 0  # refused before decoding any body
 def test_oversized_title_does_not_hide_partial_notice(monkeypatch):
    # The partial-content notice is the PR's core contract; an untrusted,
    # oversized page title must not push it past MAX_OUTPUT_CHARS.
    import asyncio
    from src.agent_tools.web_tools import WebFetchTool
    from src.constants import MAX_OUTPUT_CHARS
    def fake_fetch(url, timeout=10, max_bytes=None):
        return {
            "content": "partial body",
            "title": "T" * (MAX_OUTPUT_CHARS + 5_000),
            "error": "",
            "truncated": True,
            "fetched_bytes": WEB_FETCH_SOFT_MAX_BYTES,
            "total_bytes": 9_000_000,
        }
    import src.search.content as alias_mod
    monkeypatch.setattr(alias_mod, "fetch_webpage_content", fake_fetch)
    out = asyncio.run(WebFetchTool().execute(
        json.dumps({"url": "https://example.com/big.txt"}), ctx={}
    ))
    assert out["exit_code"] == 0
    assert out["output"].startswith("[partial content:")
    assert '"full": true' in out["output"]
 def test_tool_layer_emits_partial_notice_and_parses_full(monkeypatch):
    import asyncio
    from src.agent_tools.web_tools import WebFetchTool
    calls = {}
    def fake_fetch(url, timeout=10, max_bytes=None):
        calls["max_bytes"] = max_bytes
        return {
            "content": "partial body",
            "title": "Big File",
            "error": "",
            "truncated": True,
            "fetched_bytes": WEB_FETCH_SOFT_MAX_BYTES,
            "total_bytes": 5_000_000,
        }
    import src.search.content as alias_mod
    monkeypatch.setattr(alias_mod, "fetch_webpage_content", fake_fetch)
    out = asyncio.run(WebFetchTool().execute(
        json.dumps({"url": "https://example.com/big.txt"}), ctx={}
    ))
    assert out["exit_code"] == 0
    assert "[partial content:" in out["output"]
    assert '"full": true' in out["output"]
    assert calls["max_bytes"] is None
    asyncio.run(WebFetchTool().execute(
        json.dumps({"url": "https://example.com/big.txt", "full": True}), ctx={}
    ))
    assert calls["max_bytes"] == WEB_FETCH_HARD_MAX_BYTES
@@ -0,0 +1,54 @@
 """Guard: every public webhook emitter goes through the manager.
 Public emitters in `routes/` must schedule their fire through
 `webhook_manager.fire_and_forget(...)` (or `_spawn_tracked`). A bare
 `asyncio.create_task(webhook_manager.fire(...))` escapes
 `WebhookManager._bg_tasks`, so asyncio only holds a weak reference to the
 delivery task and the GC can collect it before it sends — silently dropping
 the webhook. Catching this with a scan stops a regression from sneaking
 back in via a copy-paste.
 """
 import ast
 from pathlib import Path
 ROUTES_DIR = Path(__file__).resolve().parent.parent / "routes"
 def _untracked_fire_calls(tree: ast.AST) -> list[tuple[int, str]]:
    """Return (lineno, snippet) for any asyncio.create_task(webhook_manager.fire(...))."""
    hits: list[tuple[int, str]] = []
    for node in ast.walk(tree):
        if not isinstance(node, ast.Call):
            continue
        func = node.func
        if not (isinstance(func, ast.Attribute) and func.attr == "create_task"):
            continue
        if not (isinstance(func.value, ast.Name) and func.value.id == "asyncio"):
            continue
        if not node.args:
            continue
        inner = node.args[0]
        if not isinstance(inner, ast.Call):
            continue
        inner_func = inner.func
        if (
            isinstance(inner_func, ast.Attribute)
            and inner_func.attr == "fire"
            and isinstance(inner_func.value, ast.Name)
            and inner_func.value.id == "webhook_manager"
        ):
            hits.append((node.lineno, ast.unparse(node)))
    return hits
 def test_no_untracked_webhook_fire_in_routes():
    offenders: list[str] = []
    for path in ROUTES_DIR.rglob("*.py"):
        tree = ast.parse(path.read_text(), filename=str(path))
        for lineno, snippet in _untracked_fire_calls(tree):
            offenders.append(f"{path.relative_to(ROUTES_DIR.parent)}:{lineno}: {snippet}")
    assert not offenders, (
        "Public webhook emitters must use webhook_manager.fire_and_forget(...) "
        "so the delivery task is tracked in WebhookManager._bg_tasks. Found "
        "untracked emitter(s):\n  " + "\n  ".join(offenders)
    )
@@ -0,0 +1,574 @@
 #!/usr/bin/env python3
 """Build the oversized test-file split plan for issue #3983.
 The output is a planning document only. It does not move tests, rewrite
 assertions, extract helpers, or change CI.
 """
 from __future__ import annotations
 import ast
 import json
 import os
 import re
 import subprocess
 import sys
 from collections import Counter
 from dataclasses import dataclass
 from pathlib import Path
 ROOT = Path(__file__).resolve().parents[2]
 TESTS_DIR = ROOT / "tests"
 OUTPUT = TESTS_DIR / "OVERSIZED_TEST_SPLIT_PLAN.md"
 RAW_OUTPUT = Path("/tmp/oversized-test-file-metrics.json")
 LARGE_LINE_THRESHOLD = 300
 LARGE_NODE_THRESHOLD = 20
 TOP_LIMIT = 30
 HIGH_RISK_SIGNALS = {"route/api", "db/session", "import-state", "security"}
@dataclass(frozen=True)
 class FileMetric:
    path: str
    lines: int
    nonblank: int
    test_defs: int
    test_classes: int
    collected: int
    area: str
    sub_area: str
    signals: tuple[str, ...]
 def read_text(path: Path) -> str:
    return path.read_text(encoding="utf-8", errors="replace")
 def count_ast_tests(text: str) -> tuple[int, int]:
    tree = ast.parse(text)
    test_defs = 0
    test_classes = 0
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
            if node.name.startswith("test_"):
                test_defs += 1
        elif isinstance(node, ast.ClassDef):
            if node.name.startswith("Test"):
                test_classes += 1
    return test_defs, test_classes
 def load_taxonomy_classifier():
    sys.path.insert(0, str(ROOT))
    from tests._taxonomy import classify_test_path
    return classify_test_path
 def classify(path: Path, classify_test_path) -> tuple[str, str]:
    rel_path = Path(path.relative_to(ROOT).as_posix())
    try:
        result = classify_test_path(rel_path)
    except Exception:
        return "unknown", "unknown"
    return getattr(result, "area", "unknown"), getattr(result, "sub_area", "unknown")
 def collect_node_counts() -> Counter[str]:
    cmd = [
        sys.executable,
        "-m",
        "pytest",
        "--collect-only",
        "-q",
        "tests",
    ]
    env = dict(os.environ)
    env["PY_COLORS"] = "0"
    result = subprocess.run(
        cmd,
        cwd=ROOT,
        env=env,
        text=True,
        capture_output=True,
    )
    if result.returncode != 0:
        print(result.stdout)
        print(result.stderr, file=sys.stderr)
        raise SystemExit(result.returncode)
    counts: Counter[str] = Counter()
    for line in result.stdout.splitlines():
        line = line.strip()
        if "::" not in line:
            continue
        if not line.startswith("tests/"):
            continue
        file_path = line.split("::", 1)[0]
        counts[file_path] += 1
    return counts
 def detect_signals(text: str, path: str) -> tuple[str, ...]:
    signal_patterns = {
        "route/api": [
            r"\bTestClient\b",
            r"\bapp\.",
            r"\broutes\.",
            r"\bfrom routes\b",
            r"\bimport routes\b",
        ],
        "db/session": [
            r"\bSessionLocal\b",
            r"\bsqlite\b",
            r"\bDATABASE_URL\b",
            r"\bcore\.database\b",
            r"\bdb\.query\b",
            r"\bcommit\(",
        ],
        "import-state": [
            r"\bsys\.modules\b",
            r"\bimportlib\b",
            r"\bclear_module\b",
            r"\bpreserve_import_state\b",
            r"\bmonkeypatch\.setitem\b",
        ],
        "security": [
            r"\bsecurity\b",
            r"\bssrf\b",
            r"\bpath traversal\b",
            r"\bcsrf\b",
            r"\bpermission\b",
        ],
        "filesystem": [
            r"\btmp_path\b",
            r"\bTemporaryDirectory\b",
            r"\bPath\(",
            r"\bmkdir\b",
            r"\bwrite_text\b",
            r"\bread_text\b",
        ],
        "subprocess/script": [
            r"\bsubprocess\b",
            r"\brunpy\b",
            r"\bload_script\b",
            r"\bsys\.argv\b",
        ],
        "async/threading": [
            r"\basyncio\b",
            r"\bthreading\b",
            r"\bconcurrent\.futures\b",
            r"\bThreadPoolExecutor\b",
        ],
        "ui/static": [
            r"\bstatic/",
            r"\bjsdom\b",
            r"\bnode\b",
            r"\.js\b",
        ],
    }
    signals = []
    for name, patterns in signal_patterns.items():
        if any(re.search(pattern, text, flags=re.IGNORECASE) for pattern in patterns):
            signals.append(name)
    if path.startswith("tests/cli/"):
        signals.append("cli-directory")
    return tuple(signals)
 def metric_for(path: Path, node_counts: Counter[str], classify_test_path) -> FileMetric:
    rel = path.relative_to(ROOT).as_posix()
    text = read_text(path)
    lines = len(text.splitlines())
    nonblank = sum(1 for line in text.splitlines() if line.strip())
    test_defs, test_classes = count_ast_tests(text)
    area, sub_area = classify(path, classify_test_path)
    return FileMetric(
        path=rel,
        lines=lines,
        nonblank=nonblank,
        test_defs=test_defs,
        test_classes=test_classes,
        collected=node_counts.get(rel, 0),
        area=area,
        sub_area=sub_area,
        signals=detect_signals(text, rel),
    )
 def test_files() -> list[Path]:
    return sorted(TESTS_DIR.rglob("test_*.py"))
 def as_metric_row(metric: FileMetric) -> str:
    signals = ", ".join(metric.signals) if metric.signals else "-"
    return (
        f"| `{metric.path}` | {metric.lines} | {metric.collected} | "
        f"{metric.test_defs} | {metric.test_classes} | "
        f"{metric.area} | {metric.sub_area} | {signals} |"
    )
 def metric_table(title: str, metrics: list[FileMetric]) -> list[str]:
    lines = [
        f"## {title}",
        "",
        "| File | Lines | Collected tests | Test defs | Test classes | Area | Sub-area | Signals |",
        "|---|---:|---:|---:|---:|---|---|---|",
    ]
    lines.extend(as_metric_row(metric) for metric in metrics)
    lines.append("")
    return lines
 def candidate_metrics(metrics: list[FileMetric]) -> list[FileMetric]:
    return [
        metric
        for metric in metrics
        if metric.lines >= LARGE_LINE_THRESHOLD
        or metric.collected >= LARGE_NODE_THRESHOLD
    ]
 def include_reasons(metric: FileMetric) -> str:
    reasons = []
    if metric.lines >= LARGE_LINE_THRESHOLD:
        reasons.append(f"{metric.lines} lines")
    if metric.collected >= LARGE_NODE_THRESHOLD:
        reasons.append(f"{metric.collected} collected tests")
    return ", ".join(reasons)
 def risk_notes(metric: FileMetric) -> str:
    if not metric.signals:
        return "No obvious setup signals from static scan."
    return ", ".join(metric.signals)
 def suggested_handling(metric: FileMetric) -> str:
    if HIGH_RISK_SIGNALS.intersection(metric.signals):
        return "Defer mechanical split until setup/risk boundaries are mapped."
    if metric.collected >= LARGE_NODE_THRESHOLD:
        return "Good first manual-review candidate if test themes are cohesive."
    return "Plan split boundaries before editing."
 def candidate_section(metrics: list[FileMetric]) -> list[str]:
    lines = [
        "## Split planning candidates",
        "",
        "This section is generated from metrics, not from manual judgement.",
        "Files are included when they meet at least one threshold:",
        "",
        f"- at least {LARGE_LINE_THRESHOLD} physical lines; or",
        f"- at least {LARGE_NODE_THRESHOLD} collected pytest items.",
        "",
        "These are planning candidates only. A later split PR still needs a focused manual review of each file before moving tests.",
        "",
        "| File | Why included | Setup/risk signals | Suggested handling |",
        "|---|---|---|---|",
    ]
    for metric in metrics:
        lines.append(
            f"| `{metric.path}` | {include_reasons(metric)} | "
            f"{risk_notes(metric)} | {suggested_handling(metric)} |"
        )
    lines.append("")
    return lines
 def first_manual_review_section(metrics: list[FileMetric]) -> list[str]:
    low_risk = [
        metric
        for metric in metrics
        if metric.area != "uncategorized"
        and not HIGH_RISK_SIGNALS.intersection(metric.signals)
    ]
    low_risk = sorted(low_risk, key=lambda m: (m.collected, m.lines), reverse=True)
    lines = [
        "## Suggested first manual-review candidates",
        "",
        "These are not automatic split approvals. They are categorized candidates with enough size/collection value and no route/API, DB/session, import-state, or security signal from the static scan.",
        "",
        "Files still in the `uncategorized` taxonomy area are listed separately below so taxonomy review does not get mixed into the first split decision.",
        "",
        "| File | Lines | Collected tests | Area | Sub-area | Signals | Why this is a candidate |",
        "|---|---:|---:|---|---|---|---|",
    ]
    if not low_risk:
        lines.append("| _None_ | - | - | - | - | - | - |")
    for metric in low_risk[:10]:
        signals = ", ".join(metric.signals) if metric.signals else "-"
        lines.append(
            f"| `{metric.path}` | {metric.lines} | {metric.collected} | "
            f"{metric.area} | {metric.sub_area} | {signals} | {include_reasons(metric)} |"
        )
    lines.append("")
    return lines
 def taxonomy_gap_section(metrics: list[FileMetric]) -> list[str]:
    uncategorized = [
        metric
        for metric in metrics
        if metric.area == "uncategorized"
    ]
    uncategorized = sorted(
        uncategorized,
        key=lambda m: (m.collected, m.lines),
        reverse=True,
    )
    lines = [
        "## Taxonomy coverage gaps among split candidates",
        "",
        "`uncategorized` is a current taxonomy area, not a builder failure.",
        "This plan does not reclassify tests because taxonomy changes should be reviewed separately from oversized-file split planning.",
        "",
        "Before using any of these files as a split target, first decide whether the taxonomy should be refined in a separate focused issue/PR.",
        "",
        "| File | Lines | Collected tests | Sub-area | Signals | Suggested follow-up |",
        "|---|---:|---:|---|---|---|",
    ]
    if not uncategorized:
        lines.append("| _None_ | - | - | - | - | - |")
    for metric in uncategorized:
        signals = ", ".join(metric.signals) if metric.signals else "-"
        follow_up = "Review taxonomy mapping before using as a split target."
        if HIGH_RISK_SIGNALS.intersection(metric.signals):
            follow_up = "Review taxonomy and setup/risk boundaries before any split."
        lines.append(
            f"| `{metric.path}` | {metric.lines} | {metric.collected} | "
            f"{metric.sub_area} | {signals} | {follow_up} |"
        )
    lines.append("")
    return lines
 def deferred_section(metrics: list[FileMetric]) -> list[str]:
    deferred = [
        metric
        for metric in metrics
        if HIGH_RISK_SIGNALS.intersection(metric.signals)
    ]
    deferred = sorted(deferred, key=lambda m: (m.collected, m.lines), reverse=True)
    lines = [
        "## High-risk candidates to defer first",
        "",
        "These files may still be split later, but not as the first implementation slice without a separate manual boundary review.",
        "",
        "| File | Lines | Collected tests | High-risk signals |",
        "|---|---:|---:|---|",
    ]
    for metric in deferred[:15]:
        signals = ", ".join(sorted(HIGH_RISK_SIGNALS.intersection(metric.signals)))
        lines.append(
            f"| `{metric.path}` | {metric.lines} | {metric.collected} | {signals} |"
        )
    lines.append("")
    return lines
 def write_distribution(
    lines: list[str],
    title: str,
    values: Counter[str],
    *,
    min_count: int = 1,
 ) -> None:
    displayed = [
        (value, count)
        for value, count in sorted(values.items())
        if count >= min_count
    ]
    omitted_values = sum(1 for count in values.values() if count < min_count)
    omitted_files = sum(count for count in values.values() if count < min_count)
    lines.extend([
        f"{title}:",
        "",
        "| Value | Files |",
        "|---|---:|",
    ])
    for value, count in displayed:
        lines.append(f"| {value} | {count} |")
    if omitted_values:
        lines.extend([
            "",
            f"Values below {min_count} files: {omitted_values} values covering {omitted_files} files.",
        ])
    lines.append("")
 def write_report(metrics: list[FileMetric], node_count_total: int) -> None:
    by_lines = sorted(metrics, key=lambda m: (m.lines, m.collected), reverse=True)
    by_collected = sorted(metrics, key=lambda m: (m.collected, m.lines), reverse=True)
    candidates = sorted(
        candidate_metrics(metrics),
        key=lambda m: (m.collected, m.lines),
        reverse=True,
    )
    areas = Counter(metric.area for metric in metrics)
    sub_areas = Counter(metric.sub_area for metric in metrics)
    lines = [
        "# Oversized Test File Split Plan",
        "",
        "## Purpose",
        "",
        "This document plans future oversized test-file splits using current repo data.",
        "It does not move files, rewrite assertions, extract helpers, or change CI.",
        "",
        "## Roadmap context",
        "",
        "- Issue: #3983",
        "- Parent tracker: #2523",
        "- Follows #3973 / #3982, the report-only order-sensitivity diagnostics slice.",
        "",
        "## Methodology",
        "",
        "Metrics were generated from the current test tree using:",
        "",
        "- physical line counts for every recursive `test_*.py` file under `tests/`;",
        "- AST counts for `test_*` functions and `Test*` classes;",
        "- one `pytest --collect-only -q tests` run to count collected items per file;",
        "- current taxonomy classification from `tests._taxonomy.classify_test_path`; and",
        "- static setup-signal scans for route/API, DB/session, import-state, security, filesystem, subprocess/script, async/threading, and UI/static indicators.",
        "",
        "Static signals are not proof of risk. They are review prompts.",
        "Future split PRs must still inspect each file manually before editing.",
        "",
        "## Current summary",
        "",
        f"- test files scanned: {len(metrics)}",
        f"- collected pytest items counted: {node_count_total}",
        f"- large-file threshold: {LARGE_LINE_THRESHOLD} lines",
        f"- large-collected threshold: {LARGE_NODE_THRESHOLD} collected items",
        "",
    ]
    write_distribution(lines, "Area distribution", areas)
    write_distribution(lines, "Sub-area distribution", sub_areas, min_count=2)
    lines.extend(metric_table("Top files by collected pytest items", by_collected[:TOP_LIMIT]))
    lines.extend(metric_table("Top files by physical line count", by_lines[:TOP_LIMIT]))
    lines.extend(candidate_section(candidates))
    lines.extend(taxonomy_gap_section(candidates))
    lines.extend(first_manual_review_section(candidates))
    lines.extend(deferred_section(candidates))
    lines.extend([
        "## Rules for future split PRs",
        "",
        "- One file or one coherent file-family per PR.",
        "- No assertion rewrites mixed with file moves.",
        "- No helper extraction mixed with file moves.",
        "- No production code changes.",
        "- No CI workflow changes.",
        "- Preserve existing markers and taxonomy unless the split issue explicitly says otherwise.",
        "- Validate the original file's collected tests before and after the split.",
        "- Validate any neighboring taxonomy/focused-runner behavior if paths change.",
        "- Treat files with route/API, DB/session, import-state, or security signals as higher-risk until manually reviewed.",
        "",
        "## Suggested next step",
        "",
        "Use this plan to choose the first actual oversized-file split issue.",
        "The first split should prefer a file with high review value and low setup risk.",
        "Do not start a split PR from this planning issue alone if the file's boundaries are still ambiguous.",
        "",
        "## Reproduction command",
        "",
        "This document was generated with:",
        "",
        "```bash",
        ".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
        "```",
        "",
        "## Freshness check",
        "",
        "After editing the builder or rebasing the branch, regenerate the plan and confirm no unexpected plan drift:",
        "",
        "```bash",
        ".venv/bin/python tests/tools/build_oversized_test_split_plan.py",
        "git diff --exit-code -- tests/OVERSIZED_TEST_SPLIT_PLAN.md",
        "```",
        "",
    ])
    OUTPUT.write_text("\n".join(lines), encoding="utf-8")
 def write_raw(metrics: list[FileMetric]) -> None:
    raw = [
        {
            "area": metric.area,
            "collected": metric.collected,
            "lines": metric.lines,
            "nonblank": metric.nonblank,
            "path": metric.path,
            "signals": list(metric.signals),
            "sub_area": metric.sub_area,
            "test_classes": metric.test_classes,
            "test_defs": metric.test_defs,
        }
        for metric in metrics
    ]
    RAW_OUTPUT.write_text(json.dumps(raw, indent=2, sort_keys=True), encoding="utf-8")
 def assert_taxonomy_worked(metrics: list[FileMetric]) -> None:
    if not metrics:
        raise SystemExit("ERROR: no test files were scanned")
    unknown = sum(1 for metric in metrics if metric.area == "unknown")
    if unknown == len(metrics):
        raise SystemExit("ERROR: taxonomy classification returned unknown for every file")
 def main() -> int:
    if not TESTS_DIR.exists():
        print("ERROR: tests/ directory not found", file=sys.stderr)
        return 1
    classify_test_path = load_taxonomy_classifier()
    node_counts = collect_node_counts()
    metrics = [metric_for(path, node_counts, classify_test_path) for path in test_files()]
    assert_taxonomy_worked(metrics)
    write_report(metrics, sum(node_counts.values()))
    write_raw(metrics)
    print(f"Wrote {OUTPUT.relative_to(ROOT)}")
    print(f"Wrote {RAW_OUTPUT}")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())