21 Commits

Author SHA1 Message Date
NoodleLDS 170560fbcc fix(logging): avoid logging raw URLs in llm_core error paths
Drop the raw url/base_chat_url from the Ollama-detection and
model-list-fetch warning logs added by this sweep, since these values
can contain private hostnames, internal IPs, credentials, or other
deployment details.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 17:31:43 +01:00
NoodleLDS 58bbbf1c1f log: pass exception via exc_info instead of string interpolation 2026-06-15 17:31:43 +01:00
NoodleLDS 48eb2abd8e log(llm_core): add warnings to silent except Exception blocks
- Malformed URL in _is_ollama_native_url now logs a warning so bad
  endpoint configs are traceable instead of silently returning False.
- Model list fetch failure now logs a warning with the endpoint URL so
  endpoints that silently vanish from the model picker are diagnosable.
2026-06-15 17:31:43 +01:00
NoodleLDS b12fe2afb6 log(agent_loop): add warnings to silent except Exception blocks
- builtin tool overrides load failure now logs a warning so misconfigured
  settings don't silently fall back to defaults without a trace.
- Timezone context injection failure now logs a warning to help debug
  incorrect scheduled times in agent-created tasks.
- PDF form-backed document detection failure now logs a warning so
  broken form-doc UI is traceable to the root cause.
2026-06-15 17:31:43 +01:00
NoodleLDS 4b6efbd11d log(document_routes): add warnings to silent except Exception blocks
- Export ZIP request body parse failure now logs a warning so empty
  exports caused by malformed requests are diagnosable.
- clear_active_document failure on detach now logs a warning to help
  trace doc re-injection bugs like #1160.
2026-06-15 17:31:42 +01:00
NoodleLDS 6b5feb0e64 log(email_routes): add warnings to silent except Exception blocks
- Email alias resolution failure now logs a warning instead of silently
  returning an empty list, making broken account configs diagnosable.
2026-06-15 17:31:42 +01:00
NoodleLDS 7d1abdbe69 log(chat_routes): add warnings to silent except Exception blocks
- clear_orphaned_session_endpoint: log before rollback so failures
  appear in traces when users see stale/deleted model options.
- _endpoint_has_model (JSON parse): log malformed cached_models instead
  of silently treating endpoint as valid.
- _has_any_visible_model (JSON parse): log malformed cached_models
  instead of silently returning empty list.
- timezone header parse: log failure so time-zone-related tool bugs
  (wrong scheduled times, calendar events) are traceable.
- attachments JSON parse: log failure so silently-dropped attachments
  are visible in server logs.
2026-06-15 17:31:42 +01:00
NoodleLDS 5ddef4af3e log(app): add warnings to silent except Exception blocks
- Internal tool auth header failure now logs a warning instead of
  silently passing, making auth bypass easier to spot in logs.
- Token last_used_at update failure now logs at DEBUG (fire-and-forget,
  non-critical, but useful when debugging token tracking issues).
- Image ownership verification failure now logs a warning so unexpected
  access-check errors surface instead of silently allowing the request.
2026-06-15 17:31:42 +01:00
Hriday Ranka 270b8570fc feat(email): add Google OAuth2 for Google Workspace / .edu IMAP & SMTP (#237)
* feat(email): add Google OAuth2 for Google Workspace / .edu IMAP & SMTP

Google deprecated basic-auth (password) access for Google Workspace
accounts in May 2025. This means any .edu or org Google email account
could no longer connect via IMAP/SMTP with a username + password —
the email feature was silently broken for a large class of users.

This PR adds full OAuth2 (XOAUTH2) support for Google accounts so
Workspace / .edu emails work out of the box.

## What changed

### Backend
- `core/database.py`: add `oauth_provider`, `oauth_access_token`,
  `oauth_refresh_token`, `oauth_token_expiry`, and `display_name`
  columns to `EmailAccount` + idempotent migration
- `routes/email_helpers.py`: XOAUTH2 auth in `_imap_connect()` and
  `_send_smtp_message()`, automatic token refresh, OAuth fields in
  `_get_email_config()`
- `routes/email_routes.py`: OAuth authorize + callback routes,
  `_smtp_ready()` fix, OAuth fields through `_deliver()` closure,
  `display_name` in `From:` header

### Frontend
- `static/js/settings.js`: "Google Workspace / .edu" provider preset,
  "Connect with Google" button, success/error banner, display name field
- `static/js/document.js`: `_accountCanSend()` recognises OAuth accounts
  as SMTP-capable

* security: sign OAuth state, scope callback by owner, fix quotes & logs

Addresses reviewer feedback on the email OAuth2 PR:

- OAuth state is now HMAC-SHA256 signed (keyed with the app secret from
  secret_storage) encoding account_id + owner + a random nonce, and is
  verified with constant-time comparison in the callback before any
  token write. Replaces the bare account_id state, closing the CSRF /
  state-guessing gap.
- Callback extracts the owner from the verified state and re-checks it
  against EmailAccount.owner before writing tokens, matching the
  ownership guards used elsewhere in the email routes. Single-user mode
  (owner == "") still accepts any account, consistent with
  _assert_owns_account.
- Replaced curly/smart quotes in the Name/Email/Display Name input rows
  with plain ASCII so getElementById lookups and event wiring work.
- Stripped account name, SMTP host/user, owner, and raw provider error
  text from send-config and OAuth logs; failures now surface as generic
  error codes in the redirect instead of raw exception strings.

* test(email): add OAuth2 state, _smtp_ready, and XOAUTH2 tests

Move the OAuth state sign/verify helpers out of the setup_email_routes
closure into module-level make_oauth_state/verify_oauth_state in
email_helpers.py so they can be unit-tested, then add tests/test_email_oauth.py:

- signed state round-trips account_id + owner, nonce is unique per call
- tampered account_id, forged signature, and garbage states are rejected
- _smtp_ready treats an OAuth account (no password) as send-capable, and
  still rejects host+user-only accounts with neither password nor OAuth
- _xoauth2_string / _xoauth2_bytes produce the correct SASL XOAUTH2 framing

14 new tests; existing test_security_regressions.py still passes (28).

* refactor(email): single XOAUTH2 frame helper, use RuntimeError

Polish from self-review before merge:

- Collapse the XOAUTH2 framing to one source of truth: _xoauth2_raw()
  returns the unencoded SASL string used by both the SMTP and IMAP auth
  callbacks (each library base64-encodes it), and _xoauth2_bytes() is
  just its .encode(). Removes the unused base64 _xoauth2_string helper
  and the duplicated inline frame in _send_smtp_message.
- Raise RuntimeError (not bare Exception) for the "OAuth token
  unavailable" path, matching the convention used across src/.
- Update tests accordingly.

All 14 OAuth tests + 28 security regressions pass; SMTP/IMAP XOAUTH2
verified live against a real Workspace account.

* tests(email-oauth): cover the security-sensitive OAuth paths before merge

The previous tests only exercised pure helpers (state signing, _smtp_ready,
XOAUTH2 framing). This adds coverage for the actual token-custody and
ownership behaviour, pinning the real route handlers rather than
re-implementations of their logic.

Real OAuth callback route (pulled live from setup_email_routes()):
- missing code -> generic missing_code redirect, no account id / owner in URL
- provider error -> generic google_error redirect, raw error not echoed
- tampered/invalid state -> invalid_state redirect, auth code never leaked
- signed state with owner mismatch -> token write refused (ownership_error),
  DB row left untouched
- signed state with matching owner -> tokens written encrypted, and only to
  the intended account (a second account stays untouched)

Real accounts-list route:
- exposes oauth_provider status but never the access/refresh token values,
  encrypted or otherwise

Token storage / refresh helpers (isolated in-memory SQLite, mocked HTTP):
- refreshed access token stored encrypted; expiry is a timestamp, not a token
- fresh token uses cache (no refresh call); expired token triggers refresh
- refresh HTTP failure returns None silently, no exception or secret surfaced
- missing client credentials short-circuits to None

Password-account regression:
- password IMAP accounts call conn.login(); OAuth accounts call XOAUTH2
  authenticate() and never login()

28 tests pass (14 prior + 14 new).

* fix(email-oauth): drop raw exception text from token-refresh log

Google token refresh failures now log the account id only, matching
the conservative logging used elsewhere on the OAuth path — no raw
provider/exception details surfacing in logs.

* fix(email-oauth): bring OAuth UI parity to the Integrations email form

The Google Workspace / .edu provider preset, Display Name field, and
Connect-with-Google flow were only wired into the Email-tab account
form. The Integrations-tab form (a separate code path for the same
account type) was missing all three, so the OAuth option was invisible
from that entry point. Mirrors the same PROVIDERS entry, OAuth section,
and connect handler so both forms behave identically.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
2026-06-15 17:02:58 +01:00
Léo 0750486654 fix(notes): fail closed when an unauthenticated request reaches owner-scoped routes (#4062)
* fix(notes): fail closed when an unauthenticated request reaches owner-scoped routes

The notes CRUD routes resolved the acting user with bare get_current_user().
A request that reached them with no identity (auth-middleware regression,
SSRF from a sibling service) came through as user=None — which every query
treats as the single-user mode: list all accounts' notes, read/update/
delete/pin/archive any row, reorder globally.

Resolve the owner through require_user() instead, which already encodes the
right policy: 401 when auth is configured, while the documented anonymous
modes (AUTH_ENABLED=false, LOCALHOST_BYPASS on loopback, unconfigured
first-run) still resolve to the single-user path. fire-reminder in the same
file already gated this way; the CRUD routes now match, and the inline
require_user import there is folded into the module import.

Extracted from #2940 (stabilization slice).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(notes): drive fail-closed test via ASGITransport, not sync TestClient

The focused fail-closed test hung at `TestClient(app).get(...)` on some
environments. Starlette's sync TestClient runs the app in a background
event-loop thread (anyio blocking portal) and then dispatches each sync
endpoint onto a second worker thread; that handshake deadlocks on certain
anyio/httpx/platform combos. The identity injection also used
BaseHTTPMiddleware (@app.middleware("http")), the other known TestClient
deadlock source.

Switch to the repo's existing httpx.ASGITransport + AsyncClient idiom so the
whole request runs on the test's own event loop (no portal thread, no
BaseHTTPMiddleware). Identity now comes from a pure-ASGI shim that writes the
same request.state fields the real auth middleware sets, and a non-loopback
client peer keeps require_user's loopback fall-throughs out of the picture.
Same assertions and coverage; production code unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 17:43:28 +02:00
RaresKeY d38e2cbc07 fix(ci): avoid duplicate CodeQL setup (#4297) 2026-06-15 16:39:13 +01:00
Ashvin 7fd937fa57 fix(calendar): parse "mins"/"hrs" reminder offsets in manage_calendar (#4266)
_reminder_minutes matched the offset with (?:m|min|minute|minutes)\b and
(?:h|hr|hour|hours)\b. The trailing \b makes the common plural
abbreviations "mins"/"hrs" fail to match (after "min" the "s" is a word
char, so no boundary), so reminder_minutes "5 mins" or "2 hrs" returned
None and the event was created with no reminder, silently.

Widen the two unit regexes and the matching reminder_only description
regex to a strict superset that also accepts mins/hrs. The sibling
duration parser already accepts these forms (it has no \b), so this only
brings the reminder parser in line.
2026-06-15 17:37:28 +02:00
Catalin Iliescu c41caac438 fix(cookbook): only persist successfully stopped scheduled serves (#4267)
Co-authored-by: Cata <cata@bigjohn.local>
2026-06-15 17:30:18 +02:00
Kenny Van de Maele 1747c13133 test: align README presentation guards with the #4306 refresh (#4311)
* test: align README presentation guards with the #4306 refresh

The 'Refresh README presentation' change (#4306) swapped the ASCII banner
for a centered wordmark image and moved the native quickstart into
docs/setup.md, which left four base tests failing on dev and froze the
merge gate:

- test_security_regressions::test_readme_native_quickstart_uses_loopback
  now also accepts the loopback guidance from docs/setup.md, where the
  quickstart moved (no behaviour change; the guidance is intact there).
- test_readme_ascii_fenced guards the new wordmark title instead of the
  removed ASCII banner, and keeps a defensive check that any reintroduced
  box-drawing banner stays inside a code fence (the original #1390 mode).
- The five unreferenced demo gifs under docs/ (chat, compare, document,
  notes, research) are removed so test_docs_no_orphan_images passes; they
  were de-referenced by the refresh. Recoverable from history if a docs
  page wants to embed them again.

* chore: refresh PR checks

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-06-15 16:25:38 +01:00
RaresKeY ffd0aaf69b fix(cookbook): validate adopt host (#4282) 2026-06-15 16:44:24 +02:00
RaresKeY 81e7074d93 fix(gallery): confine replacement image path (#4285) 2026-06-15 16:42:41 +02:00
RaresKeY f66a23d19d fix(ai): validate generated image result URLs (#4289) 2026-06-15 16:40:49 +02:00
RaresKeY f602819523 fix(models): scope API-token model listing (#4292) 2026-06-15 16:38:41 +02:00
RaresKeY 85a773ea02 fix(personal): resolve upload delete path (#4291) 2026-06-15 16:38:37 +02:00
PewDiePie fb0a64fe4f Merge pull request #4306 from pewdiepie-archdaemon/readme-refresh-default-branch
docs: refresh README presentation
2026-06-15 23:28:20 +09:00
pewdiepie-archdaemon bcf46dafb9 Refresh README presentation 2026-06-15 23:26:10 +09:00
35 changed files with 1819 additions and 109 deletions
+6 -7
View File
@@ -331,8 +331,8 @@ if AUTH_ENABLED:
request.state.current_user = "internal-tool"
request.state.api_token = False
return await call_next(request)
except Exception:
pass
except Exception as _e:
logger.warning("Internal tool auth header check failed", exc_info=_e)
# Allow DIRECT localhost requests (internal service calls from
# heartbeats etc.). Tunnel/proxy-forwarded requests are excluded by
# _is_trusted_loopback so LOCALHOST_BYPASS can't be abused over a
@@ -385,11 +385,10 @@ if AUTH_ENABLED:
_db.close()
try:
await _asyncio.to_thread(_do)
except Exception:
pass
except Exception as _e:
logger.debug("Failed to update token last_used_at", exc_info=_e)
_asyncio.create_task(_touch_last_used(matched_id))
# Keep bearer-token callers out of normal cookie/user
# routes. API-aware routes can read api_token_owner.
request.state.current_user = "api"
request.state.api_token = True
request.state.api_token_id = matched_id
@@ -464,8 +463,8 @@ async def serve_generated_image(filename: str, request: Request):
_db.close()
except HTTPException:
raise
except Exception:
pass
except Exception as _e:
logger.warning("Image ownership verification failed for %r", filename, exc_info=_e)
ext = filename.rsplit('.', 1)[-1].lower()
mime = {
"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg",
+27
View File
@@ -324,6 +324,13 @@ class EmailAccount(TimestampMixin, Base):
smtp_password = Column(String, default="")
from_address = Column(String, default="")
display_name = Column(String, nullable=True) # "Hriday Ranka" — used in From: header
# OAuth2 (Google / Google Workspace). Tokens stored encrypted via secret_storage.
oauth_provider = Column(String, nullable=True) # "google" or None
oauth_access_token = Column(String, nullable=True) # encrypted
oauth_refresh_token = Column(String, nullable=True) # encrypted
oauth_token_expiry = Column(String, nullable=True) # unix timestamp string
__table_args__ = (
Index('ix_email_accounts_owner_default', 'owner', 'is_default'),
@@ -1427,6 +1434,25 @@ def _migrate_add_task_automation_columns():
except Exception as e:
logging.getLogger(__name__).warning(f"task automation migration: {e}")
def _migrate_add_email_oauth_columns():
"""Add Google OAuth and display_name columns to email_accounts if missing."""
try:
with engine.connect() as conn:
cols = [r[1] for r in conn.execute(text("PRAGMA table_info(email_accounts)"))]
for col, typedef in [
("oauth_provider", "TEXT"),
("oauth_access_token", "TEXT"),
("oauth_refresh_token", "TEXT"),
("oauth_token_expiry", "TEXT"),
("display_name", "TEXT"),
]:
if col not in cols:
conn.execute(text(f"ALTER TABLE email_accounts ADD COLUMN {col} {typedef}"))
conn.commit()
except Exception as e:
logging.getLogger(__name__).warning(f"email oauth columns migration: {e}")
def _migrate_add_oauth_config():
"""Add oauth_config column to mcp_servers table if missing."""
try:
@@ -1771,6 +1797,7 @@ def init_db():
_migrate_add_tidy_verdict()
_migrate_add_doc_source_email_cols()
_migrate_add_oauth_config()
_migrate_add_email_oauth_columns()
_migrate_add_task_automation_columns()
_migrate_add_disabled_tools()
_migrate_add_mcp_oauth_tokens_column()
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.0 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.4 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1003 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.5 MiB

+14 -9
View File
@@ -1,14 +1,16 @@
# Security CI guide
This project runs a set of automated security checks on every pull request and
on every push to `main`. This page explains what each one does, whether it can
This project runs a set of automated security checks on pull requests and
selected branch pushes. This page explains what each one does, whether it can
block a merge, and the few one-time settings you should turn on to get the full
benefit.
## What runs, and why
Each check lives in its own file under `.github/workflows/`. They run
automatically; you do not start them.
Most checks live in files under `.github/workflows/`. CodeQL is configured
through GitHub's code scanning default setup, so it appears as a dynamic GitHub
workflow instead of a checked-in workflow file. They run automatically; you do
not start them.
| Check | What it protects against | Blocks a merge? |
|---|---|---|
@@ -88,11 +90,14 @@ let the workflows run on one pull request first, then add them here.
2. Turn on **Dependency graph** (usually on by default for public repos) -- this
powers Dependency review and Dependabot.
3. Turn on **Dependabot alerts** and **Dependabot security updates**.
4. Under **Code scanning**, you have two ways to scan the app code with CodeQL:
- The included `codeql.yml` workflow already scans `main` and runs weekly.
- To also scan **pull requests** (recommended, since most contributions come
from forks), click **Set up -> Default** under Code scanning. GitHub then
runs CodeQL on pull requests for you, with no token limitations.
4. Under **Code scanning**, use **Set up -> Default** for CodeQL. GitHub then
runs CodeQL as a dynamic workflow without the fork-token limitations that
affect checked-in advanced workflows.
Do not also add a checked-in CodeQL workflow while default setup is enabled:
GitHub rejects advanced CodeQL uploads when default setup is active. If the
project later needs an advanced CodeQL workflow, disable default setup first
and keep only one CodeQL publishing path active.
## Keeping it current
+8 -5
View File
@@ -126,7 +126,8 @@ def _clear_orphaned_session_endpoint(sess, owner: str | None = None) -> bool:
sess.model = ""
sess.headers = {}
return True
except Exception:
except Exception as e:
logger.warning("Failed to clear orphaned session endpoint", exc_info=e)
db.rollback()
return False
finally:
@@ -144,7 +145,8 @@ def _endpoint_cache_contains_model(endpoint, model: str) -> bool:
return True
try:
models = json.loads(raw) if isinstance(raw, str) else raw
except Exception:
except Exception as e:
logger.warning("Failed to parse cached models list, treating as containing model", exc_info=e)
return True
if not isinstance(models, list) or not models:
return True
@@ -236,7 +238,8 @@ def _recover_empty_session_model(sess, session_id: str, owner: str | None = None
is_chatgpt_subscription = False
try:
cached = json.loads(ep.cached_models) if isinstance(ep.cached_models, str) else (ep.cached_models or [])
except Exception:
except Exception as e:
logger.warning("Failed to parse cached_models for endpoint %r", getattr(ep, "id", "?"), exc_info=e)
cached = []
if not cached:
visible = []
@@ -646,8 +649,8 @@ def setup_chat_routes(
elif attachments:
try:
att_ids = [str(x) for x in json.loads(attachments)]
except Exception:
pass
except Exception as e:
logger.warning("Failed to parse attachments JSON, ignoring attachments", exc_info=e)
no_memory = str(form_data.get("no_memory", "")).lower() == "true"
pre_context_tool_policy = build_effective_tool_policy(
+1 -1
View File
@@ -790,7 +790,7 @@ def setup_codex_routes(
norm = dict(body or {})
sess = (norm.get("tmux_session") or norm.get("session_id") or "").strip()
model = (norm.get("model") or norm.get("repo_id") or "").strip()
host = (norm.get("host") or norm.get("remote_host") or "").strip()
host = validate_remote_host((norm.get("host") or norm.get("remote_host") or "").strip() or None) or ""
port = norm.get("port") or 8000
import re as _re
if not sess or not _re.fullmatch(r"[a-zA-Z0-9_-]+", sess):
+4 -3
View File
@@ -503,7 +503,8 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
user = get_current_user(request)
try:
data = await request.json()
except Exception:
except Exception as e:
logger.warning("Failed to parse export request body, defaulting to empty", exc_info=e)
data = {}
ids = data.get("ids") or []
if not ids:
@@ -645,8 +646,8 @@ def setup_document_routes(session_manager, upload_handler=None) -> APIRouter:
try:
from src.agent_tools.document_tools import clear_active_document
clear_active_document(doc_id)
except Exception:
pass
except Exception as e:
logger.warning("Failed to clear active document %r on detach", doc_id, exc_info=e)
db.commit()
db.refresh(doc)
return _doc_to_dict(doc)
+134 -10
View File
@@ -13,6 +13,8 @@ and `email_pollers.py` (the background loops):
"""
import os
import base64
import time
import imaplib
import smtplib
import email as email_mod
@@ -38,6 +40,106 @@ from src.secret_storage import decrypt as _decrypt
logger = logging.getLogger(__name__)
def _xoauth2_raw(user: str, access_token: str) -> str:
"""The SASL XOAUTH2 initial-response string (unencoded).
Both smtplib.SMTP.auth() and imaplib.IMAP4.authenticate() base64-encode
the value their callback returns, so callers pass this raw form never
pre-encoded to avoid double base64.
"""
return f"user={user}\x01auth=Bearer {access_token}\x01\x01"
def _xoauth2_bytes(user: str, access_token: str) -> bytes:
"""Raw XOAUTH2 bytes for imaplib's authenticate() callback."""
return _xoauth2_raw(user, access_token).encode()
def make_oauth_state(account_id: str, owner: str) -> str:
"""Return an HMAC-signed, base64-encoded OAuth state token.
Encodes account_id + owner + a random nonce, signed with the app secret
so the callback can validate that the flow was initiated by an
authenticated, owning user (CSRF / state-forgery protection).
"""
import hmac as _hmac, hashlib as _hl, secrets as _sec
from src.secret_storage import _load_or_create_key
nonce = _sec.token_hex(16)
payload = json.dumps({"a": account_id, "o": owner, "n": nonce}, separators=(",", ":"))
sig = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
return base64.urlsafe_b64encode(f"{payload}|{sig}".encode()).decode()
def verify_oauth_state(state: str) -> dict | None:
"""Verify an OAuth state token's HMAC signature.
Returns the decoded payload dict ({"a", "o", "n"}) on success, or None if
the token is malformed, tampered, or signed with a different key.
"""
import hmac as _hmac, hashlib as _hl
from src.secret_storage import _load_or_create_key
try:
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload, sig = decoded.rsplit("|", 1)
expected = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
if not _hmac.compare_digest(sig, expected):
return None
return json.loads(payload)
except Exception:
return None
def _refresh_google_token(account_id: str) -> str | None:
"""Exchange the stored refresh token for a new access token and persist it."""
import httpx
from core.database import SessionLocal as _SL, EmailAccount as _EA
from src.secret_storage import encrypt as _enc, decrypt as _dec
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
if not client_id or not client_secret:
return None
db = _SL()
try:
row = db.get(_EA, account_id)
if not row or not row.oauth_refresh_token:
return None
refresh_token = _dec(row.oauth_refresh_token or "")
if not refresh_token:
return None
resp = httpx.post("https://oauth2.googleapis.com/token", data={
"client_id": client_id,
"client_secret": client_secret,
"refresh_token": refresh_token,
"grant_type": "refresh_token",
}, timeout=10)
resp.raise_for_status()
data = resp.json()
access_token = data["access_token"]
row.oauth_access_token = _enc(access_token)
row.oauth_token_expiry = str(int(time.time()) + data.get("expires_in", 3600))
db.commit()
return access_token
except Exception:
logger.warning(f"Google token refresh failed for account {account_id}")
return None
finally:
db.close()
def _get_valid_google_token(account_id: str, cfg: dict) -> str | None:
"""Return a valid Google access token, refreshing if expired or missing."""
from src.secret_storage import decrypt as _dec
access_token = _dec(cfg.get("oauth_access_token") or "")
expiry_str = cfg.get("oauth_token_expiry") or ""
if access_token and expiry_str:
try:
if int(expiry_str) - 60 > time.time():
return access_token
except (ValueError, TypeError):
pass
return _refresh_google_token(account_id)
def _smtp_security_mode(cfg: dict) -> str:
raw = str(cfg.get("smtp_security") or "").strip().lower()
if raw in {"ssl", "starttls", "none"}:
@@ -54,20 +156,29 @@ def _send_smtp_message(cfg: dict, from_addr: str, recipients: list[str], message
port = int(cfg.get("smtp_port") or 465)
user = cfg.get("smtp_user") or ""
password = cfg.get("smtp_password") or ""
def _auth_smtp(smtp):
if cfg.get("oauth_provider") == "google":
token = _get_valid_google_token(cfg.get("account_id"), cfg)
if not token:
raise RuntimeError("Google OAuth token unavailable — reconnect the account")
smtp.ehlo()
smtp.auth("XOAUTH2", lambda challenge=None: _xoauth2_raw(user, token), initial_response_ok=True)
elif user and password:
smtp.login(user, password)
security = _smtp_security_mode(cfg)
if security == "ssl":
with smtplib.SMTP_SSL(host, port, timeout=timeout) as smtp:
if user and password:
smtp.login(user, password)
_auth_smtp(smtp)
smtp.sendmail(from_addr, recipients, message)
return
with smtplib.SMTP(host, port, timeout=timeout) as smtp:
if security == "starttls":
smtp.starttls()
if user and password:
smtp.login(user, password)
_auth_smtp(smtp)
smtp.sendmail(from_addr, recipients, message)
@@ -701,10 +812,16 @@ def _get_email_config(account_id: str | None = None, owner: str = "") -> dict:
"imap_password": _decrypt(row.imap_password or ""),
"imap_starttls": bool(row.imap_starttls),
"from_address": row.from_address or row.imap_user or "",
"oauth_provider": row.oauth_provider or "",
"oauth_access_token": row.oauth_access_token or "",
"oauth_refresh_token": row.oauth_refresh_token or "",
"oauth_token_expiry": row.oauth_token_expiry or "",
"display_name": row.display_name or "",
}
if not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
is_oauth = bool(cfg.get("oauth_provider"))
if not is_oauth and not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
logger.warning(f"SMTP not configured for account {row.name!r}")
if not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
if not is_oauth and not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
logger.warning(f"IMAP not configured for account {row.name!r}")
return cfg
finally:
@@ -825,12 +942,19 @@ def _imap_connect(account_id: str | None = None, owner: str = "",
timeout=timeout,
)
try:
conn.login(cfg["imap_user"], cfg["imap_password"])
if cfg.get("oauth_provider") == "google":
token = _get_valid_google_token(cfg.get("account_id"), cfg)
if not token:
raise RuntimeError("Google OAuth token unavailable — reconnect the account in Settings → Integrations")
conn.authenticate("XOAUTH2", lambda x: _xoauth2_bytes(cfg["imap_user"], token))
else:
conn.login(cfg["imap_user"], cfg["imap_password"])
except Exception:
# A failed AUTHENTICATE (e.g. an Office 365 app password on an
# MFA-enabled tenant, #3174) otherwise orphans the already-connected
# socket; close it before propagating so a misconfigured account
# can't leak one descriptor per retry / background poller pass.
# MFA-enabled tenant, #3174, or an expired/revoked OAuth token)
# otherwise orphans the already-connected socket; close it before
# propagating so a misconfigured account can't leak one descriptor
# per retry / background poller pass.
try:
conn.shutdown()
except Exception:
+146 -8
View File
@@ -13,7 +13,9 @@ handlers need. The split is mechanical — no behavior change.
"""
import asyncio
import os
import sqlite3 as _sql3
import time
import email as email_mod
import email.header
import email.utils
@@ -43,6 +45,7 @@ from routes.email_helpers import (
_load_settings, _save_settings, _get_email_config,
_send_smtp_message, _smtp_security_mode,
_IMAP_TIMEOUT_SECONDS, _open_imap_connection,
make_oauth_state, verify_oauth_state,
_imap_connect, _imap, _decode_header, _detect_sent_folder, _detect_drafts_folder,
_extract_attachment_text, _list_attachments_from_msg,
_extract_attachment_to_disk, _extract_html, _extract_text,
@@ -76,15 +79,16 @@ def _email_tag_owner_aliases(account_id: str | None, owner: str = "") -> list[st
cfg.get("smtp_user") or "",
cfg.get("from_address") or "",
])
except Exception:
except Exception as _e:
logger.warning("Failed to resolve email account alias", exc_info=_e)
resolved_account_id = None
row = db.get(_EA, resolved_account_id) if resolved_account_id else None
if row:
aliases.extend([row.owner or "", row.imap_user or "", row.from_address or ""])
finally:
db.close()
except Exception:
pass
except Exception as _e:
logger.warning("Failed to load email aliases", exc_info=_e)
out = []
for a in aliases:
a = (a or "").strip()
@@ -285,7 +289,9 @@ def _group_uid_fetch_records(msg_data) -> list:
def _smtp_ready(cfg: dict) -> bool:
return bool(cfg.get("smtp_host") and cfg.get("smtp_user") and cfg.get("smtp_password"))
if not cfg.get("smtp_host") or not cfg.get("smtp_user"):
return False
return bool(cfg.get("smtp_password") or cfg.get("oauth_provider"))
def _resolve_send_config(account_id: str | None = None, owner: str = "") -> dict:
@@ -2021,7 +2027,7 @@ def setup_email_routes():
outer = MIMEMultipart("alternative")
body_container = outer
outer["From"] = cfg["from_address"]
outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
outer["To"] = to
if cc:
outer["Cc"] = cc
@@ -2285,6 +2291,7 @@ def setup_email_routes():
try:
cfg = _resolve_send_config(req.account_id, owner=owner)
except Exception as e:
logger.warning(f"No SMTP-capable account resolved: {e}")
return {"success": False, "error": str(e) or "No SMTP-capable email account configured"}
# Use 'mixed' if we have attachments, 'alternative' otherwise
@@ -2297,7 +2304,7 @@ def setup_email_routes():
outer = MIMEMultipart("alternative")
body_container = outer
outer["From"] = cfg["from_address"]
outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
outer["To"] = req.to
if req.cc:
outer["Cc"] = req.cc
@@ -2348,6 +2355,10 @@ def setup_email_routes():
_account_id = cfg.get("account_id") or req.account_id # capture for the IMAP append in the closure
_in_reply_to = (req.in_reply_to or "").strip()
_oauth_provider = cfg.get("oauth_provider") or ""
_oauth_access_token = cfg.get("oauth_access_token") or ""
_oauth_refresh_token = cfg.get("oauth_refresh_token") or ""
_oauth_token_expiry = cfg.get("oauth_token_expiry") or ""
def _deliver():
try:
@@ -2358,6 +2369,11 @@ def setup_email_routes():
"smtp_security": _smtp_security,
"smtp_user": _smtp_user,
"smtp_password": _smtp_pw,
"account_id": _account_id,
"oauth_provider": _oauth_provider,
"oauth_access_token": _oauth_access_token,
"oauth_refresh_token": _oauth_refresh_token,
"oauth_token_expiry": _oauth_token_expiry,
},
_from,
_recipients,
@@ -2470,7 +2486,7 @@ def setup_email_routes():
msg.attach(MIMEText(_draft_html, "html", "utf-8"))
else:
msg = MIMEText(req.body, "plain", "utf-8")
msg["From"] = cfg["from_address"]
msg["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
msg["To"] = req.to
if req.cc:
msg["Cc"] = req.cc
@@ -3122,6 +3138,8 @@ def setup_email_routes():
"from_address": r.from_address or "",
"has_imap_password": bool(r.imap_password),
"has_smtp_password": bool(r.smtp_password),
"oauth_provider": r.oauth_provider or "",
"display_name": r.display_name or "",
})
return {"accounts": out}
finally:
@@ -3154,6 +3172,7 @@ def setup_email_routes():
smtp_user=(data.get("smtp_user") or "").strip(),
smtp_password=_enc(data.get("smtp_password") or ""),
from_address=(data.get("from_address") or "").strip(),
display_name=(data.get("display_name") or "").strip(),
# SECURITY: stamp the creator so all subsequent reads / mutations
# can filter by user. Without this every new account leaks to
# every other user.
@@ -3188,7 +3207,7 @@ def setup_email_routes():
if not row:
return {"ok": False, "error": "Account not found"}
# Simple fields
for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address"):
for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address", "display_name"):
if key in data:
setattr(row, key, (data[key] or "").strip())
for key in ("imap_port", "smtp_port"):
@@ -3377,4 +3396,123 @@ def setup_email_routes():
finally:
db.close()
# ── Google OAuth2 routes ──
@router.get("/oauth/google/authorize")
async def google_oauth_authorize(account_id: str = Query(...), request: Request = None, owner: str = Depends(require_user)):
import urllib.parse
_assert_owns_account(account_id, owner)
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
if not client_id:
raise HTTPException(400, "GOOGLE_OAUTH_CLIENT_ID not set — add it to .env")
redirect_uri = (
os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
)
state = make_oauth_state(account_id, owner)
params = urllib.parse.urlencode({
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": "https://mail.google.com/ email",
"access_type": "offline",
"prompt": "consent",
"state": state,
})
from fastapi.responses import RedirectResponse as _RR
return _RR(f"https://accounts.google.com/o/oauth2/v2/auth?{params}")
@router.get("/oauth/google/callback")
async def google_oauth_callback(
code: str = Query(None),
state: str = Query(None),
error: str = Query(None),
request: Request = None,
):
import urllib.parse
from fastapi.responses import RedirectResponse as _RR
if error:
return _RR("/?section=integrations&email_oauth_error=google_error")
if not code or not state:
return _RR("/?section=integrations&email_oauth_error=missing_code")
state_data = verify_oauth_state(state)
if not state_data:
return _RR("/?section=integrations&email_oauth_error=invalid_state")
account_id = state_data.get("a", "")
owner = state_data.get("o", "")
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
redirect_uri = (
os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
)
import httpx as _httpx
try:
resp = _httpx.post("https://oauth2.googleapis.com/token", data={
"code": code,
"client_id": client_id,
"client_secret": client_secret,
"redirect_uri": redirect_uri,
"grant_type": "authorization_code",
}, timeout=10)
resp.raise_for_status()
data = resp.json()
except Exception:
logger.warning("Google token exchange failed")
return _RR("/?section=integrations&email_oauth_error=token_exchange_failed")
access_token = data.get("access_token", "")
refresh_token = data.get("refresh_token", "")
expiry = str(int(time.time()) + data.get("expires_in", 3600))
# Fetch the email address from userinfo so we can auto-fill imap_user.
email_addr = ""
display_name = ""
try:
ui = _httpx.get("https://www.googleapis.com/oauth2/v1/userinfo",
headers={"Authorization": f"Bearer {access_token}"}, timeout=10)
if ui.is_success:
ui_data = ui.json()
email_addr = ui_data.get("email", "")
display_name = ui_data.get("name", "")
except Exception:
pass
from core.database import SessionLocal, EmailAccount
from src.secret_storage import encrypt as _enc
db = SessionLocal()
try:
row = db.query(EmailAccount).filter(EmailAccount.id == account_id).first()
if not row:
return _RR("/?section=integrations&email_oauth_error=account_not_found")
# SECURITY: verify the account belongs to the initiating user.
if owner and row.owner and row.owner != owner:
logger.warning("OAuth callback owner mismatch — rejecting token write")
return _RR("/?section=integrations&email_oauth_error=ownership_error")
row.oauth_provider = "google"
row.oauth_access_token = _enc(access_token)
if refresh_token:
row.oauth_refresh_token = _enc(refresh_token)
row.oauth_token_expiry = expiry
# Auto-fill Google IMAP/SMTP settings if not already configured.
if not row.imap_host:
row.imap_host = "imap.gmail.com"
row.imap_port = 993
row.imap_starttls = False
if not row.smtp_host:
row.smtp_host = "smtp.gmail.com"
row.smtp_port = 587
if email_addr:
if not row.imap_user:
row.imap_user = email_addr
if not row.smtp_user:
row.smtp_user = email_addr
if not row.from_address:
row.from_address = email_addr
if not row.name or row.name == row.id:
row.name = email_addr
if display_name and not row.display_name:
row.display_name = display_name
db.commit()
finally:
db.close()
return _RR("/?section=integrations&email_oauth_success=1")
return router
+2 -5
View File
@@ -232,8 +232,6 @@ def setup_gallery_routes() -> APIRouter:
@router.post("/api/gallery/{image_id}/replace")
async def gallery_replace(request: Request, image_id: str):
"""Replace an existing gallery image file with a new one."""
from pathlib import Path
user = get_current_user(request)
db = SessionLocal()
try:
@@ -249,9 +247,8 @@ def setup_gallery_routes() -> APIRouter:
raise HTTPException(400, "No image provided")
content = await read_upload_limited(file, GALLERY_UPLOAD_MAX_BYTES, "Gallery replacement")
img_dir = Path(GENERATED_IMAGES_DIR)
img_dir.mkdir(parents=True, exist_ok=True)
img_path = img_dir / _sanitize_gallery_filename(img.filename)
GALLERY_IMAGE_DIR.mkdir(parents=True, exist_ok=True)
img_path = _gallery_image_path(img.filename)
img_path.write_bytes(content)
# Refresh dimensions in case the editor resized the canvas.
+11 -8
View File
@@ -26,7 +26,7 @@ from src.endpoint_resolver import (
build_models_url,
build_headers,
)
from src.auth_helpers import _auth_disabled, owner_filter
from src.auth_helpers import _auth_disabled, effective_user, owner_filter
logger = logging.getLogger(__name__)
@@ -1255,13 +1255,16 @@ def setup_model_routes(model_discovery):
# Require auth; "" is the unconfigured single-user mode, treated as
# "see everything" by _fetch_models.
try:
from src.auth_helpers import get_current_user as _gcu
owner = _gcu(request) or ""
except Exception:
owner = ""
# Reject anonymous in configured deployments — no leaking the model
# list to unauthenticated callers.
try:
if getattr(request.state, "api_token", False):
scopes = set(getattr(request.state, "api_token_scopes", []) or [])
if "chat" not in scopes:
raise HTTPException(403, "API token is not scoped for chat")
if not getattr(request.state, "api_token_owner", None):
raise HTTPException(403, "API token has no owner")
owner = effective_user(request) or ""
# Reject anonymous in configured deployments — no leaking the model
# list to unauthenticated callers.
auth_mgr = getattr(request.app.state, "auth_manager", None)
if not owner and not _auth_disabled() and auth_mgr is not None and getattr(auth_mgr, "is_configured", False):
raise HTTPException(401, "Not authenticated")
+12 -4
View File
@@ -10,7 +10,7 @@ from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from core.database import SessionLocal, Note
from src.auth_helpers import get_current_user
from src.auth_helpers import require_user
from src.constants import DATA_DIR
from sqlalchemy.orm.attributes import flag_modified
@@ -570,7 +570,16 @@ def setup_note_routes(task_scheduler=None):
router = APIRouter(prefix="/api/notes", tags=["notes"])
def _owner(request: Request) -> Optional[str]:
return get_current_user(request)
# require_user, not bare get_current_user: a request that reaches
# these owner-scoped routes with NO identity (auth-middleware
# regression, SSRF from a sibling service) must fail closed (401)
# when auth is configured — not be treated as the single-user mode
# and handed blanket access to every account's notes. The documented
# anonymous modes (AUTH_ENABLED=false, LOCALHOST_BYPASS on loopback,
# unconfigured first-run) still resolve to None, the single-user
# path. fire_reminder below already gated this way; the CRUD routes
# did not.
return require_user(request) or None
def _is_admin_or_single_user(request: Request, user: str | None) -> bool:
if user == "internal-tool":
@@ -805,8 +814,7 @@ def setup_note_routes(task_scheduler=None):
Returns {synthesis, email_sent}.
"""
# Gate against anonymous callers — LLM synthesis can burn tokens.
from src.auth_helpers import require_user as _ru
user = _ru(request)
user = require_user(request)
body = await request.json()
note_id = str(body.get("note_id") or "").strip()
if not note_id:
+2 -2
View File
@@ -278,8 +278,8 @@ def setup_personal_routes(personal_docs_manager, rag_manager, rag_available):
# Delete file from disk if it's in uploads dir
deleted_from_disk = False
try:
abs_target = os.path.abspath(filepath)
base_abs = os.path.abspath(UPLOADS_DIR)
abs_target = os.path.realpath(filepath)
base_abs = os.path.realpath(UPLOADS_DIR)
in_uploads = (
abs_target == base_abs
or os.path.commonpath([abs_target, base_abs]) == base_abs
+5 -5
View File
@@ -524,7 +524,7 @@ def get_builtin_overrides() -> dict:
ov = get_setting("builtin_tool_overrides", {})
return ov if isinstance(ov, dict) else {}
except Exception as e:
logger.warning('Failed to load builtin tool overrides: %s', e)
logger.warning("Failed to load builtin tool overrides, using defaults", exc_info=e)
return {}
@@ -929,8 +929,8 @@ def _build_system_prompt(
try:
from src.user_time import current_datetime_context_message
_datetime_message = current_datetime_context_message()
except Exception:
pass
except Exception as e:
logger.warning("Failed to build datetime context message", exc_info=e)
# Document context is kept as a SEPARATE message (not merged into the tool
# prompt) so the context trimmer doesn't destroy it when truncating the
@@ -973,8 +973,8 @@ def _build_system_prompt(
try:
from src.pdf_form_doc import find_source_upload_id
_is_form_backed = bool(find_source_upload_id(active_document.current_content or ""))
except Exception:
pass
except Exception as e:
logger.warning("Failed to detect if document is form-backed, assuming plain", exc_info=e)
if _is_form_backed:
doc_ctx = (
+12 -3
View File
@@ -1613,7 +1613,9 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
"""
import base64
import httpx
import os
from pathlib import Path
from src.url_safety import check_outbound_url
lines = content.strip().split("\n")
prompt = lines[0].strip() if lines else ""
@@ -1779,8 +1781,15 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
elif img.get("url"):
# Download external URL and save locally (DALL-E returns temp URLs)
result_url = img["url"]
ok, reason = check_outbound_url(
result_url,
block_private=os.getenv("IMAGE_BLOCK_PRIVATE_IPS", "false").lower() == "true",
)
if not ok:
return {"error": f"Image API returned unsafe image URL: {reason}"}
try:
dl_resp = httpx.get(img["url"], timeout=60)
dl_resp = httpx.get(result_url, timeout=60)
if dl_resp.status_code == 200:
img_dir = Path(GENERATED_IMAGES_DIR)
img_dir.mkdir(parents=True, exist_ok=True)
@@ -1790,10 +1799,10 @@ async def do_generate_image(content: str, session_id: Optional[str] = None, owne
image_url = f"/api/generated-image/{filename}"
image_id = _save_to_gallery(filename)
else:
image_url = img["url"] # fallback to external URL
image_url = result_url # fallback to external URL
except Exception as _dl_e:
logger.warning(f"Failed to download DALL-E image: {_dl_e}")
image_url = img["url"] # fallback to external URL
image_url = result_url # fallback to external URL
else:
return {"error": "Image API returned unexpected format (no b64_json or url)"}
+3 -2
View File
@@ -161,11 +161,13 @@ async def _tick() -> None:
# Re-read state once before writing so we capture any updates from
# concurrent UI syncs.
stopped_any = False
successfully_stopped_sids = set()
for sid, host, port in to_stop:
ok = await _stop_serve(sid, host, port)
logger.info(f"cookbook_serve_lifecycle: stop {sid} (host={host or 'local'}): {'ok' if ok else 'failed'}")
if ok:
stopped_any = True
successfully_stopped_sids.add(sid)
# Drop the auto-registered endpoint so the model picker and
# the chat router don't keep pointing at a dead server.
for t in tasks:
@@ -188,12 +190,11 @@ async def _tick() -> None:
except Exception:
fresh = state
fresh_tasks = tasks
stopped_sids = {sid for sid, _, _ in to_stop}
for ft in fresh_tasks:
if not isinstance(ft, dict):
continue
ft_sid = ft.get("sessionId") or ft.get("id")
if ft_sid in stopped_sids:
if ft_sid in successfully_stopped_sids:
ft["status"] = "stopped"
ft["_scheduledStopAtMs"] = None
ft["_lastStatusFlipAt"] = now_ms
+4 -3
View File
@@ -283,7 +283,8 @@ def _is_ollama_native_url(url: str) -> bool:
"""Return True for native Ollama API URLs, including Ollama Cloud."""
try:
parsed = urlparse(url or "")
except Exception:
except Exception as e:
logger.warning("Failed to parse URL for Ollama detection", exc_info=e)
return False
host = parsed.hostname or ""
path = (parsed.path or "").rstrip("/")
@@ -1345,8 +1346,8 @@ def list_model_ids(
r = httpx.get(root + "/api/tags", timeout=timeout)
r.raise_for_status()
return [m.get("name") or m.get("model") for m in (r.json().get("models") or []) if m.get("name") or m.get("model")]
except Exception:
pass
except Exception as e:
logger.warning("Failed to fetch model list from configured endpoint", exc_info=e)
return []
def normalize_model_id(
+3 -3
View File
@@ -1579,10 +1579,10 @@ async def do_manage_calendar(content: str, owner: Optional[str] = None) -> Dict:
text = str(raw).strip().lower()
if text in {"none", "no", "off", "false"}:
return None
m = re.search(r"(\d+)\s*(?:m|min|minute|minutes)\b", text)
m = re.search(r"(\d+)\s*(?:minutes?|mins?|m)\b", text)
if m:
return max(0, int(m.group(1)))
m = re.search(r"(\d+)\s*(?:h|hr|hour|hours)\b", text)
m = re.search(r"(\d+)\s*(?:hours?|hrs?|h)\b", text)
if m:
return max(0, int(m.group(1)) * 60)
if text.isdigit():
@@ -1595,7 +1595,7 @@ async def do_manage_calendar(content: str, owner: Optional[str] = None) -> Dict:
return desc
reminder_only = re.compile(
r"^\s*(?:remind(?:er)?|alarm)\s*:?\s*\d+\s*"
r"(?:m|min|minute|minutes|h|hr|hour|hours)\b.*$",
r"(?:minutes?|mins?|m|hours?|hrs?|h)\b.*$",
re.I,
)
return "" if reminder_only.match(desc) else desc
+2 -1
View File
@@ -87,7 +87,8 @@ import * as Modals from './modalManager.js';
}
function _accountCanSend(account) {
return !!(account && account.smtp_host && account.smtp_user && account.has_smtp_password);
if (!account || !account.smtp_host || !account.smtp_user) return false;
return !!(account.has_smtp_password || account.oauth_provider);
}
async function _resolveComposeSendAccountId() {
+128 -10
View File
@@ -2913,13 +2913,14 @@ async function initEmailAccountsSettings() {
// IMAP and SMTP. Dovecot is IMAP-only here; the host is intentionally
// blank because it may live on another machine (DNS, LAN, Tailscale).
const PROVIDERS = {
gmail: { label: 'Gmail', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
migadu: { label: 'Migadu', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
fastmail: { label: 'Fastmail', imap: { host: 'imap.fastmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com', port: 465 } },
yahoo: { label: 'Yahoo', imap: { host: 'imap.mail.yahoo.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com', port: 465 } },
dovecot: { label: 'Dovecot IMAP (no SMTP)', imap: { host: '', port: 31143, starttls: false }, smtp: { host: '', port: 465 } },
gmail: { label: 'Gmail', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
google_workspace: { label: 'Google Workspace / .edu', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 587 }, oauth: 'google' },
migadu: { label: 'Migadu', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
fastmail: { label: 'Fastmail', imap: { host: 'imap.fastmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com', port: 465 } },
yahoo: { label: 'Yahoo', imap: { host: 'imap.mail.yahoo.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com', port: 465 } },
dovecot: { label: 'Dovecot IMAP (no SMTP)', imap: { host: '', port: 31143, starttls: false }, smtp: { host: '', port: 465 } },
};
const _providerOptions = Object.entries(PROVIDERS)
.map(([k, v]) => `<option value="${k}">${esc(v.label)}</option>`)
@@ -2932,11 +2933,17 @@ async function initEmailAccountsSettings() {
<div id="eaf-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
<div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="eaf-name" class="settings-input" placeholder="(optional leave blank to use email)" value="${esc(a.name || '')}"></div>
<div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="eaf-from" class="settings-input" placeholder="you@example.com" value="${esc(a.from_address || '')}"></div>
<div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="eaf-display-name" class="settings-input" placeholder="Your Name" value="${esc(a.display_name || '')}"></div>
<div id="eaf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
<div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 required for Workspace / .edu accounts</div>
<div id="eaf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${a.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
<button type="button" id="eaf-oauth-btn" class="admin-btn-add" style="font-size:11px">${a.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
</div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:6px 0 2px">IMAP (Receiving)</div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="eaf-imap-host" class="settings-input" value="${esc(a.imap_host || '')}"></div>
<div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="eaf-imap-port" class="settings-input" type="number" value="${esc(a.imap_port || 993)}" style="max-width:100px"></div>
<div class="settings-row"><label class="settings-label">Username${_hint('Usually your full email address.')}</label><input id="eaf-imap-user" class="settings-input" value="${esc(a.imap_user || '')}"></div>
<div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div>
<div class="eaf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div></div>
<div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch"><input type="checkbox" id="eaf-imap-starttls" ${a.imap_starttls !== false ? 'checked' : ''}><span class="admin-slider"></span></label></div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px">SMTP (Sending) <span style="font-weight:normal;opacity:0.7"> optional, leave blank for read-only</span></div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com, smtp.migadu.com. Leave blank to make this account read-only.')}</label><input id="eaf-smtp-host" class="settings-input" value="${esc(a.smtp_host || '')}"></div>
@@ -2959,6 +2966,16 @@ async function initEmailAccountsSettings() {
</div>
`;
// Show/hide OAuth section and password fields based on provider selection.
function _syncOauthUI(providerKey) {
const p = PROVIDERS[providerKey];
const isOauth = !!(p && p.oauth);
el('eaf-oauth-section').style.display = isOauth ? '' : 'none';
formEl.querySelectorAll('.eaf-password-section').forEach(r => {
r.style.display = isOauth ? 'none' : '';
});
}
const eafProviderNotes = {
outlook: {
title: 'Outlook / Office 365 needs OAuth',
@@ -2983,13 +3000,41 @@ async function initEmailAccountsSettings() {
el('eaf-provider').addEventListener('change', (e) => {
_renderEafProviderNote(e.target.value);
const p = PROVIDERS[e.target.value];
if (!p) return;
if (!p) { _syncOauthUI(''); return; }
el('eaf-imap-host').value = p.imap.host;
el('eaf-imap-port').value = p.imap.port;
el('eaf-imap-starttls').checked = !!p.imap.starttls;
el('eaf-smtp-host').value = p.smtp.host;
el('eaf-smtp-port').value = p.smtp.port;
el('eaf-smtp-security').value = p.smtp.security || ((parseInt(p.smtp.port || 465) === 587) ? 'starttls' : 'ssl');
_syncOauthUI(e.target.value);
});
// Init OAuth UI for accounts already connected via OAuth.
if (a.oauth_provider === 'google') _syncOauthUI('google_workspace');
// "Connect with Google" button — save the account first, then redirect to OAuth.
el('eaf-oauth-btn').addEventListener('click', async () => {
// Must save the account first to get an account_id to pass to the OAuth flow.
const body = {
name: el('eaf-name').value.trim() || el('eaf-from').value.trim(),
from_address: el('eaf-from').value.trim(),
imap_host: el('eaf-imap-host').value.trim(),
imap_port: parseInt(el('eaf-imap-port').value) || 993,
imap_user: el('eaf-imap-user').value.trim(),
imap_starttls: el('eaf-imap-starttls').checked,
smtp_host: el('eaf-smtp-host').value.trim(),
smtp_port: parseInt(el('eaf-smtp-port').value) || 587,
smtp_user: el('eaf-imap-user').value.trim(),
};
if (!body.name) { el('eaf-msg').textContent = 'Enter a Name or Email first'; el('eaf-msg').style.color = 'var(--red)'; return; }
const url = isEdit ? `/api/email/accounts/${a.id}` : '/api/email/accounts';
const method = isEdit ? 'PUT' : 'POST';
const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
const d = await r.json();
if (!d.ok) { el('eaf-msg').textContent = d.error || 'Save failed'; el('eaf-msg').style.color = 'var(--red)'; return; }
const accId = isEdit ? a.id : d.id;
window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
});
el('eaf-smtp-security').value = _smtpSecurity(a);
@@ -3009,6 +3054,7 @@ async function initEmailAccountsSettings() {
const body = {
name: el('eaf-name').value.trim(),
from_address: el('eaf-from').value.trim(),
display_name: el('eaf-display-name').value.trim(),
imap_host: el('eaf-imap-host').value.trim(),
imap_port: parseInt(el('eaf-imap-port').value) || 993,
imap_user: el('eaf-imap-user').value.trim(),
@@ -4317,6 +4363,7 @@ async function initUnifiedIntegrations() {
// it may be remote (DNS, LAN, Tailscale), not localhost.
const PROVIDERS = {
gmail: { label: 'Gmail', emailEx: 'you@gmail.com', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
google_workspace: { label: 'Google Workspace / .edu', emailEx: 'you@yourschool.edu', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 587 }, oauth: 'google' },
migadu: { label: 'Migadu', emailEx: 'you@yourdomain.com', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', emailEx: 'you@icloud.com', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', emailEx: 'you@outlook.com', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
@@ -4334,6 +4381,7 @@ async function initUnifiedIntegrations() {
const PROV_LOGO = {
'': _customLogo,
gmail: _letterLogo('G', '#ea4335'),
google_workspace: _letterLogo('G', '#ea4335'),
migadu: _letterLogo('M', '#3aa39d'),
icloud: _letterLogo('i', '#3693f3'),
outlook: _letterLogo('O', '#0078d4'),
@@ -4362,11 +4410,17 @@ async function initUnifiedIntegrations() {
<div id="uf-email-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
<div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="uf-email-name" class="settings-input" placeholder="(optional leave blank to use email)"></div>
<div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="uf-email-from" class="settings-input" placeholder="you@example.com"></div>
<div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="uf-display-name" class="settings-input" placeholder="Your Name"></div>
<div id="uf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
<div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 required for Workspace / .edu accounts</div>
<div id="uf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${existing && existing.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
<button type="button" id="uf-oauth-btn" class="admin-btn-add" style="font-size:11px">${existing && existing.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
</div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:4px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><polyline points="22 12 16 12 14 15 10 15 8 12 2 12"/><path d="M5.45 5.11 2 12v6a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-6l-3.45-6.89A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z"/></svg>IMAP (Receiving)</div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="uf-imap-host" class="settings-input" placeholder="imap.example.com"></div>
<div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="uf-imap-port" class="settings-input" type="number" placeholder="993" style="max-width:100px"></div>
<div class="settings-row"><label class="settings-label">Username${_hint('Yes — your full email address goes here too (e.g. you@gmail.com). Same as the Email field above for almost every provider.')}</label><input id="uf-imap-user" class="settings-input" placeholder="you@example.com"></div>
<div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div>
<div class="uf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div></div>
<div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch" style="margin-left:0"><input type="checkbox" id="uf-imap-starttls" checked><span class="admin-slider"></span></label></div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><line x1="22" y1="2" x2="11" y2="13"/><polygon points="22 2 15 22 11 13 2 9 22 2"/></svg>SMTP (Sending) <span style="font-weight:normal;opacity:0.7"> optional, leave blank for read-only</span></div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com. Leave blank to make this account read-only.')}</label><input id="uf-smtp-host" class="settings-input" placeholder="smtp.example.com"></div>
@@ -4491,6 +4545,16 @@ async function initUnifiedIntegrations() {
</div>`;
};
// Show/hide the OAuth section and password fields based on provider selection.
function _syncOauthUI(providerKey) {
const p = PROVIDERS[providerKey];
const isOauth = !!(p && p.oauth);
el('uf-oauth-section').style.display = isOauth ? '' : 'none';
formEl.querySelectorAll('.uf-password-section').forEach(r => {
r.style.display = isOauth ? 'none' : '';
});
}
// Custom dropdown wire-up — the native <select> stays in the DOM as the
// data source and accessibility target, but the visible UI is a button +
// popup so each provider row can render with its SVG logo. Selecting an
@@ -4547,6 +4611,7 @@ async function initUnifiedIntegrations() {
el('uf-email-provider').addEventListener('change', (e) => {
const key = e.target.value;
_renderProviderNote(key);
_syncOauthUI(key);
const p = PROVIDERS[key];
if (!p) return;
el('uf-imap-host').value = p.imap.host;
@@ -4562,6 +4627,23 @@ async function initUnifiedIntegrations() {
}
});
// Init OAuth UI for accounts already connected via OAuth.
if (existing && existing.oauth_provider === 'google') _syncOauthUI('google_workspace');
// "Connect with Google" — save the account first, then redirect to OAuth.
el('uf-oauth-btn').addEventListener('click', async () => {
const body = _collectBody();
if (!body.name) body.name = body.from_address;
if (!body.name) { el('uf-email-msg').textContent = 'Enter a Name or Email first'; el('uf-email-msg').style.color = 'var(--red)'; return; }
const url = isEdit ? `/api/email/accounts/${editId}` : '/api/email/accounts';
const method = isEdit ? 'PUT' : 'POST';
const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
const d = await r.json();
if (!(d.ok || d.id)) { el('uf-email-msg').textContent = d.error || 'Save failed'; el('uf-email-msg').style.color = 'var(--red)'; return; }
const accId = isEdit ? editId : d.id;
window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
});
// "Same as IMAP" toggle — hide the SMTP creds rows when on.
const _syncSmtpSame = () => {
const same = el('uf-smtp-same').checked;
@@ -4574,6 +4656,7 @@ async function initUnifiedIntegrations() {
if (existing) {
el('uf-email-name').value = existing.name || '';
el('uf-email-from').value = existing.from_address || '';
el('uf-display-name').value = existing.display_name || '';
el('uf-imap-host').value = existing.imap_host || '';
el('uf-imap-port').value = existing.imap_port || 993;
el('uf-imap-user').value = existing.imap_user || '';
@@ -4622,6 +4705,7 @@ async function initUnifiedIntegrations() {
const body = {
name: el('uf-email-name').value.trim(),
from_address: el('uf-email-from').value.trim(),
display_name: el('uf-display-name').value.trim(),
imap_host: el('uf-imap-host').value.trim(),
imap_port: parseInt(el('uf-imap-port').value) || 993,
imap_user: el('uf-imap-user').value.trim(),
@@ -5650,6 +5734,40 @@ export function close() {
}
}
// Handle redirect back from Google OAuth2 — open settings to integrations and show status.
(function _handleOauthRedirect() {
const sp = new URLSearchParams(window.location.search);
if (!sp.has('email_oauth_success') && !sp.has('email_oauth_error')) return;
// Strip params from URL without a page reload.
const clean = window.location.pathname + window.location.hash;
window.history.replaceState(null, '', clean);
const success = sp.has('email_oauth_success');
const errMsg = sp.get('email_oauth_error') || '';
// Open settings → integrations after the app has initialised.
function _tryOpen() {
if (window.settingsModule && typeof window.settingsModule.open === 'function') {
window.settingsModule.open('integrations');
// Brief toast-style banner.
const banner = document.createElement('div');
banner.textContent = success
? '✓ Google account connected — email is ready'
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
Object.assign(banner.style, {
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
});
document.body.appendChild(banner);
setTimeout(() => banner.remove(), 4000);
} else {
setTimeout(_tryOpen, 100);
}
}
_tryOpen();
})();
const settingsModule = { open, close, initIntegrations, initUnifiedIntegrations, syncAdminVisibility, refreshAiModelEndpoints };
+104
View File
@@ -0,0 +1,104 @@
from src import ai_interaction
class _GenerationResponse:
status_code = 200
text = ""
def __init__(self, image_url):
self._image_url = image_url
def json(self):
return {"data": [{"url": self._image_url}]}
class _DownloadResponse:
status_code = 503
content = b""
def _patch_generation(monkeypatch, image_url):
async def _post(self, url, json, headers):
return _GenerationResponse(image_url)
class _AsyncClient:
def __init__(self, *args, **kwargs):
pass
async def __aenter__(self):
return self
async def __aexit__(self, *exc):
return False
post = _post
import httpx
import src.settings as settings
monkeypatch.setattr(settings, "load_settings", lambda: {})
monkeypatch.setattr(httpx, "AsyncClient", _AsyncClient)
monkeypatch.setattr(
ai_interaction,
"_resolve_model",
lambda model_spec, owner=None: (
"https://api.openai.example/v1/chat/completions",
"dall-e-3",
{"Authorization": "Bearer test"},
),
)
async def test_generate_image_validates_provider_url_before_download(monkeypatch):
import httpx
import src.url_safety as url_safety
provider_url = "https://images.example.com/generated.png?sig=abc"
events = []
_patch_generation(monkeypatch, provider_url)
def _check_outbound_url(url, *, block_private=False):
events.append(("check", url, block_private))
return True, "ok"
def _get(url, *, timeout):
events.append(("get", url, timeout))
return _DownloadResponse()
monkeypatch.setattr(url_safety, "check_outbound_url", _check_outbound_url)
monkeypatch.setattr(httpx, "get", _get)
result = await ai_interaction.do_generate_image("draw a chair\ndall-e-3")
assert result["image_url"] == provider_url
assert events == [
("check", provider_url, False),
("get", provider_url, 60),
]
async def test_generate_image_rejects_unsafe_provider_url_without_download(monkeypatch):
import httpx
import src.url_safety as url_safety
unsafe_url = "http://169.254.169.254/latest/meta-data"
events = []
_patch_generation(monkeypatch, unsafe_url)
def _check_outbound_url(url, *, block_private=False):
events.append(("check", url, block_private))
return False, "link-local address blocked (SSRF metadata risk): 169.254.169.254"
def _get(url, *, timeout):
raise AssertionError("unsafe provider image URL must not be downloaded")
monkeypatch.setattr(url_safety, "check_outbound_url", _check_outbound_url)
monkeypatch.setattr(httpx, "get", _get)
result = await ai_interaction.do_generate_image("draw a chair\ndall-e-3")
assert result["error"] == (
"Image API returned unsafe image URL: "
"link-local address blocked (SSRF metadata risk): 169.254.169.254"
)
assert events == [("check", unsafe_url, False)]
@@ -0,0 +1,88 @@
"""do_manage_calendar must honour abbreviated reminder phrasings like "mins"/"hrs".
`_reminder_minutes` parsed the reminder offset with regexes anchored on
`(?:m|min|minute|minutes)\b` / `(?:h|hr|hour|hours)\b`. The trailing `\b`
made the very common plural abbreviations "mins" and "hrs" fail to match
(after "min" the next char "s" is a word char, so no boundary), so a request
like ``reminder_minutes: "5 mins"`` silently produced no reminder at all
even though the sibling duration parser (no `\b`) already accepted them.
"""
import json
import sys
import uuid
import pytest
from tests.helpers.import_state import clear_fake_database_modules
from tests.helpers.sqlite_db import make_temp_sqlite
clear_fake_database_modules()
import core.database as cdb
from core.database import Note
_TS, _ENGINE, _TMPDB = make_temp_sqlite(cdb.Base.metadata)
@pytest.fixture(autouse=True)
def _bind_temp_db(monkeypatch):
monkeypatch.setitem(sys.modules, "core.database", cdb)
parent = sys.modules.get("core")
if parent is not None:
monkeypatch.setattr(parent, "database", cdb, raising=False)
monkeypatch.setattr(cdb, "SessionLocal", _TS)
yield
async def _create_with_reminder(reminder, owner):
from src.tool_implementations import do_manage_calendar
payload = {
"action": "create_event",
"summary": "Dentist",
# Far-future so the reminder is never "already passed".
"dtstart": "2030-01-01T10:00:00",
"reminder_minutes": reminder,
}
return await do_manage_calendar(json.dumps(payload), owner=owner)
@pytest.mark.parametrize("reminder,expected", [
("5 mins", 5),
("10 mins", 10),
("2 hrs", 120),
("1 hr", 60),
("15 minutes", 15), # regression: long form still works
("30m", 30), # regression: bare unit still works
])
async def test_reminder_minutes_accepts_abbreviations(reminder, expected):
owner = "tester-" + uuid.uuid4().hex[:6]
res = await _create_with_reminder(reminder, owner)
assert res.get("exit_code") == 0, res
assert f"reminder {expected} min before" in res.get("response", ""), res
db = _TS()
try:
note = (
db.query(Note)
.filter(Note.owner == owner, Note.title == "Reminder: Dentist")
.first()
)
assert note is not None, "reminder note should have been created"
finally:
db.close()
async def test_no_reminder_when_offset_absent():
owner = "tester-" + uuid.uuid4().hex[:6]
from src.tool_implementations import do_manage_calendar
payload = {
"action": "create_event",
"summary": "No Reminder Event",
"dtstart": "2030-02-01T10:00:00",
}
res = await do_manage_calendar(json.dumps(payload), owner=owner)
assert res.get("exit_code") == 0, res
assert "reminder set" not in res.get("response", ""), res
+50
View File
@@ -7,12 +7,39 @@ in ``remoteHost`` would be injected into that command.
These pin validation on the host/port before they reach the ssh string, matching
the validators the rest of the cookbook routes already apply.
"""
import asyncio
import pytest
from fastapi import HTTPException
from starlette.requests import Request
import routes.codex_routes as codex_routes
def _route_endpoint(path: str, method: str):
router = codex_routes.setup_codex_routes()
for route in router.routes:
if route.path == path and method in route.methods:
return route.endpoint
raise AssertionError(f"{method} {path} route not found")
def _launch_request() -> Request:
request = Request(
{
"type": "http",
"method": "POST",
"path": "/api/codex/cookbook/adopt",
"headers": [],
"state": {},
}
)
request.state.api_token = True
request.state.api_token_owner = "alice"
request.state.api_token_scopes = ["cookbook:launch"]
return request
def test_rejects_remote_host_with_shell_metacharacters():
task = {"remoteHost": "box; rm -rf ~", "sshPort": ""}
with pytest.raises(HTTPException) as exc:
@@ -47,3 +74,26 @@ def test_default_ssh_port_omits_flag():
)
assert host == "box"
assert port_flag == ""
def test_adopt_rejects_ssh_option_host_before_shell(monkeypatch):
calls = []
async def fail_if_shell_runs(*args, **kwargs):
calls.append((args, kwargs))
raise RuntimeError("shell should not run for invalid host")
monkeypatch.setattr(asyncio, "create_subprocess_shell", fail_if_shell_runs)
endpoint = _route_endpoint("/api/codex/cookbook/adopt", "POST")
body = {
"tmux_session": "serve_abc123",
"model": "org/model",
"host": "-oProxyCommand=sh",
}
with pytest.raises(HTTPException) as exc:
asyncio.run(endpoint(_launch_request(), body))
assert exc.value.status_code == 400
assert calls == []
+50
View File
@@ -0,0 +1,50 @@
import json
import pytest
from src import cookbook_serve_lifecycle as lifecycle
@pytest.mark.asyncio
async def test_tick_persists_only_successfully_stopped_serves(tmp_path, monkeypatch):
state_path = tmp_path / "cookbook_state.json"
state_path.write_text(
json.dumps({
"tasks": [
{
"id": "stop-succeeds",
"type": "serve",
"status": "running",
"_scheduledStopAtMs": 0,
},
{
"id": "stop-fails",
"type": "serve",
"status": "running",
"_scheduledStopAtMs": 0,
},
]
}),
encoding="utf-8",
)
async def fake_stop_serve(session_id, remote_host="", ssh_port=""):
return session_id == "stop-succeeds"
async def fake_delete_endpoint(task):
return None
monkeypatch.setattr(lifecycle, "COOKBOOK_STATE_FILE", str(state_path))
monkeypatch.setattr(lifecycle, "_stop_serve", fake_stop_serve)
monkeypatch.setattr(lifecycle, "_delete_endpoint_for_task", fake_delete_endpoint)
await lifecycle._tick()
tasks = {
task["id"]: task
for task in json.loads(state_path.read_text(encoding="utf-8"))["tasks"]
}
assert tasks["stop-succeeds"]["status"] == "stopped"
assert tasks["stop-succeeds"]["_scheduledStopAtMs"] is None
assert tasks["stop-fails"]["status"] == "running"
assert tasks["stop-fails"]["_scheduledStopAtMs"] == 0
+580
View File
@@ -0,0 +1,580 @@
"""Tests for the Google OAuth2 email helpers.
Covers the security-critical surface added for Google Workspace / .edu
IMAP/SMTP support:
- `make_oauth_state` / `verify_oauth_state` HMAC-signed OAuth state so the
callback can't be CSRF'd or have its account_id/owner tampered with.
- `_smtp_ready` an OAuth account (no stored password) must still count as
send-capable; a host+user-only account without password or OAuth must not.
- `_xoauth2_raw` / `_xoauth2_bytes` SASL XOAUTH2 framing for SMTP/IMAP.
- `_refresh_google_token` token refresh stores result encrypted; failure is
silent (no token/secret in logs or return value).
- `_get_valid_google_token` uses cached token when fresh; calls refresh when
expired.
- `google_oauth_callback` (real route) invalid/tampered/missing state and
provider errors return generic redirects with no PII; owner mismatch refuses
the token write; a valid owner writes encrypted tokens only to the intended
account.
- `list_email_accounts` (real route) exposes OAuth status but never token
values.
- `_imap_connect` password accounts use login(); OAuth accounts use XOAUTH2.
Route tests pull the live endpoint out of `setup_email_routes()` and call it
directly they pin the real handler, not a re-implementation. The ASGI app is
not booted; outbound HTTP is mocked and the DB is an isolated in-memory SQLite.
"""
import base64
import json
import time
import unittest.mock as mock
import pytest
# ── OAuth state signing ──────────────────────────────────────────
def test_oauth_state_round_trips_account_and_owner():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
payload = verify_oauth_state(state)
assert payload is not None
assert payload["a"] == "acct-123"
assert payload["o"] == "user@example.com"
assert payload["n"] # nonce present
def test_oauth_state_nonce_is_unique_per_call():
from routes.email_helpers import make_oauth_state, verify_oauth_state
a = verify_oauth_state(make_oauth_state("acct", "o"))
b = verify_oauth_state(make_oauth_state("acct", "o"))
assert a["n"] != b["n"]
def test_oauth_state_rejects_tampered_account_id():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload_str, sig = decoded.rsplit("|", 1)
payload = json.loads(payload_str)
payload["a"] = "evil-acct" # attacker swaps the target account
forged = base64.urlsafe_b64encode(
(json.dumps(payload, separators=(",", ":")) + "|" + sig).encode()
).decode()
assert verify_oauth_state(forged) is None
def test_oauth_state_rejects_forged_signature():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload_str, _ = decoded.rsplit("|", 1)
forged = base64.urlsafe_b64encode((payload_str + "|" + "deadbeef" * 8).encode()).decode()
assert verify_oauth_state(forged) is None
@pytest.mark.parametrize("garbage", ["", "not-base64-at-all", "###", "a|b|c"])
def test_oauth_state_rejects_garbage(garbage):
from routes.email_helpers import verify_oauth_state
assert verify_oauth_state(garbage) is None
# ── _smtp_ready: OAuth accounts have no password but can still send ──
def test_smtp_ready_true_for_oauth_account_without_password():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.gmail.com",
"smtp_user": "me@nyu.edu",
"smtp_password": "",
"oauth_provider": "google",
}
assert _smtp_ready(cfg) is True
def test_smtp_ready_true_for_password_account():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.example.com",
"smtp_user": "me@example.com",
"smtp_password": "app-password",
"oauth_provider": "",
}
assert _smtp_ready(cfg) is True
def test_smtp_ready_false_without_password_or_oauth():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.example.com",
"smtp_user": "me@example.com",
"smtp_password": "",
"oauth_provider": "",
}
assert _smtp_ready(cfg) is False
def test_smtp_ready_false_without_host():
from routes.email_routes import _smtp_ready
cfg = {"smtp_host": "", "smtp_user": "me@x.com", "oauth_provider": "google"}
assert _smtp_ready(cfg) is False
# ── XOAUTH2 SASL framing ─────────────────────────────────────────
def test_xoauth2_raw_is_unencoded_sasl_frame():
from routes.email_helpers import _xoauth2_raw
assert _xoauth2_raw("me@nyu.edu", "tok123") == "user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
def test_xoauth2_bytes_is_raw_frame_encoded():
from routes.email_helpers import _xoauth2_bytes
assert _xoauth2_bytes("me@nyu.edu", "tok123") == b"user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
# ── Helpers for in-memory DB fixtures ────────────────────────────
def _make_db():
"""Return (Session, SessionFactory) backed by an isolated in-memory SQLite DB.
Used to test DB-touching helpers without the real database.
The factory lets tests open a fresh session after the helper closes its own.
"""
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from core.database import Base
engine = create_engine("sqlite:///:memory:", connect_args={"check_same_thread": False})
Base.metadata.create_all(engine)
Factory = sessionmaker(bind=engine)
return Factory(), Factory
def _make_account(session, account_id="acct-1", owner="alice", **kwargs):
"""Insert a minimal EmailAccount row and return it."""
from core.database import EmailAccount
row = EmailAccount(
id=account_id,
owner=owner,
name=kwargs.get("name", "Test"),
from_address=kwargs.get("from_address", "test@example.com"),
imap_host=kwargs.get("imap_host", "imap.gmail.com"),
imap_port=kwargs.get("imap_port", 993),
imap_user=kwargs.get("imap_user", "test@example.com"),
smtp_host=kwargs.get("smtp_host", "smtp.gmail.com"),
smtp_port=kwargs.get("smtp_port", 587),
smtp_user=kwargs.get("smtp_user", "test@example.com"),
)
for k, v in kwargs.items():
if hasattr(row, k):
setattr(row, k, v)
session.add(row)
session.commit()
return row
# ── Token encryption at rest ─────────────────────────────────────
def test_refresh_token_stored_encrypted_not_raw():
"""_refresh_google_token must encrypt the new access token before writing it
to the DB storing the raw token string would expose credentials at rest."""
from src.secret_storage import encrypt as _enc, decrypt as _dec
from core.database import EmailAccount
raw_token = "ya29.test_access_token_raw"
db, Factory = _make_db()
_make_account(db, account_id="acct-r", owner="bob",
oauth_refresh_token=_enc("refresh-tok-xyz"))
db.close()
fake_resp = mock.MagicMock()
fake_resp.raise_for_status = mock.MagicMock()
fake_resp.json.return_value = {"access_token": raw_token, "expires_in": 3600}
with mock.patch("httpx.post", return_value=fake_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-r")
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-r").first()
stored = row.oauth_access_token
verify_db.close()
assert result == raw_token, "function should return the plain access token to callers"
assert stored != raw_token, "raw token must not be stored directly in the DB"
assert _dec(stored) == raw_token, "stored value must decrypt back to the raw token"
def test_refresh_stores_encrypted_expiry_not_token():
"""oauth_token_expiry stores only a timestamp, never the token value."""
from src.secret_storage import encrypt as _enc
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-e", owner="bob",
oauth_refresh_token=_enc("ref-tok"))
db.close()
fake_resp = mock.MagicMock()
fake_resp.raise_for_status = mock.MagicMock()
fake_resp.json.return_value = {"access_token": "ya29.secret", "expires_in": 3600}
with mock.patch("httpx.post", return_value=fake_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
_refresh_google_token("acct-e")
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-e").first()
expiry = row.oauth_token_expiry
verify_db.close()
assert "ya29" not in (expiry or ""), \
"token_expiry must be a timestamp, not the token string"
# ── Real OAuth callback route ─────────────────────────────────────
#
# These pull the actual google_oauth_callback endpoint out of the router and
# invoke it — they pin the real route's behaviour, not a re-implementation, so
# they fail if the ownership/state guards are ever removed or weakened.
def _callback_endpoint():
"""Return the live google_oauth_callback endpoint from the email router."""
from routes.email_routes import setup_email_routes
router = setup_email_routes()
for route in router.routes:
if route.path == "/api/email/oauth/google/callback" and "GET" in getattr(route, "methods", set()):
return route.endpoint
raise AssertionError("google_oauth_callback route not found")
class _FakeRequest:
"""Minimal stand-in for starlette Request — the callback only reads headers."""
headers = {"host": "localhost:7000"}
def _location(resp):
"""Pull the redirect target out of a RedirectResponse."""
return resp.headers["location"]
@pytest.mark.asyncio
async def test_callback_missing_code_returns_generic_error():
"""No `code` query param → generic error redirect, with no account id, owner,
or state echoed back into the URL."""
from routes.email_helpers import make_oauth_state
callback = _callback_endpoint()
state = make_oauth_state("acct-1", "alice")
resp = await callback(code=None, state=state, error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=missing_code" in loc
assert "acct-1" not in loc, "account id must not appear in redirect URL"
assert "alice" not in loc, "owner must not appear in redirect URL"
@pytest.mark.asyncio
async def test_callback_provider_error_returns_generic_error():
"""An `error` from Google → generic error redirect, no raw provider text."""
callback = _callback_endpoint()
resp = await callback(code=None, state=None, error="access_denied", request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=google_error" in loc
assert "access_denied" not in loc, "raw provider error must not leak into redirect"
@pytest.mark.asyncio
async def test_callback_tampered_state_returns_generic_error_no_leak():
"""Tampered/invalid state → invalid_state redirect; the auth code and any
token must never appear in the redirect URL."""
callback = _callback_endpoint()
resp = await callback(code="4/secret-auth-code", state="not-a-valid-state",
error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=invalid_state" in loc
assert "4/secret-auth-code" not in loc, "auth code must not leak into redirect"
assert "token" not in loc
@pytest.mark.asyncio
async def test_callback_owner_mismatch_does_not_write_tokens():
"""A signed, valid state whose owner does not match the target account's
owner must NOT write tokens this blocks one authenticated user from
binding their Google account onto another user's mailbox row.
"""
from routes.email_helpers import make_oauth_state
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-x", owner="alice")
db.close()
# Token-exchange + userinfo would succeed — the point is the ownership gate
# rejects the write *before* trusting them.
token_resp = mock.MagicMock()
token_resp.raise_for_status = mock.MagicMock()
token_resp.json.return_value = {"access_token": "ya29.attacker", "refresh_token": "r", "expires_in": 3600}
userinfo_resp = mock.MagicMock()
userinfo_resp.is_success = True
userinfo_resp.json.return_value = {"email": "bob@evil.com", "name": "Bob"}
# State is genuinely signed, but for owner "bob" — not the row owner "alice".
state = make_oauth_state("acct-x", "bob")
with mock.patch("httpx.post", return_value=token_resp), \
mock.patch("httpx.get", return_value=userinfo_resp), \
mock.patch("core.database.SessionLocal", Factory):
callback = _callback_endpoint()
resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=ownership_error" in loc
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-x").first()
token_after = row.oauth_access_token
verify_db.close()
assert token_after is None, "no token may be written when ownership check fails"
@pytest.mark.asyncio
async def test_callback_valid_owner_writes_encrypted_tokens_to_intended_account():
"""A signed state whose owner matches the target account writes the tokens —
and only to that account, stored encrypted (raw token never persisted)."""
from routes.email_helpers import make_oauth_state
from src.secret_storage import decrypt as _dec
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-v", owner="alice", imap_host="", smtp_host="")
_make_account(db, account_id="acct-other", owner="alice") # must stay untouched
db.close()
raw_access = "ya29.legit_access_token"
raw_refresh = "1//legit_refresh_token"
token_resp = mock.MagicMock()
token_resp.raise_for_status = mock.MagicMock()
token_resp.json.return_value = {"access_token": raw_access, "refresh_token": raw_refresh, "expires_in": 3600}
userinfo_resp = mock.MagicMock()
userinfo_resp.is_success = True
userinfo_resp.json.return_value = {"email": "alice@nyu.edu", "name": "Alice"}
state = make_oauth_state("acct-v", "alice")
with mock.patch("httpx.post", return_value=token_resp), \
mock.patch("httpx.get", return_value=userinfo_resp), \
mock.patch("core.database.SessionLocal", Factory):
callback = _callback_endpoint()
resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
assert "email_oauth_success=1" in _location(resp)
verify_db = Factory()
target = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-v").first()
other = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-other").first()
verify_db.close()
assert target.oauth_provider == "google"
assert target.oauth_access_token != raw_access, "access token must be stored encrypted"
assert _dec(target.oauth_access_token) == raw_access
assert _dec(target.oauth_refresh_token) == raw_refresh
assert other.oauth_access_token is None, "tokens must only touch the intended account"
# ── Token refresh scenarios ───────────────────────────────────────
def test_get_valid_google_token_uses_cached_when_fresh():
"""_get_valid_google_token must NOT call refresh when the stored token is
still valid (expiry - 60s buffer > now). Refresh is an outbound HTTP call
that should only happen when genuinely needed."""
from src.secret_storage import encrypt as _enc
from routes.email_helpers import _get_valid_google_token
future_expiry = str(int(time.time()) + 7200) # 2 hours from now
cfg = {
"account_id": "acct-fresh",
"oauth_access_token": _enc("ya29.fresh_token"),
"oauth_token_expiry": future_expiry,
}
with mock.patch("routes.email_helpers._refresh_google_token") as mock_refresh:
result = _get_valid_google_token("acct-fresh", cfg)
assert result == "ya29.fresh_token"
mock_refresh.assert_not_called()
def test_get_valid_google_token_refreshes_when_expired():
"""_get_valid_google_token must call refresh when the token is expired."""
from src.secret_storage import encrypt as _enc
from routes.email_helpers import _get_valid_google_token
past_expiry = str(int(time.time()) - 10) # already expired
cfg = {
"account_id": "acct-exp",
"oauth_access_token": _enc("ya29.old_token"),
"oauth_token_expiry": past_expiry,
}
with mock.patch("routes.email_helpers._refresh_google_token", return_value="ya29.new_token") as mock_refresh:
result = _get_valid_google_token("acct-exp", cfg)
mock_refresh.assert_called_once_with("acct-exp")
assert result == "ya29.new_token"
def test_refresh_failure_returns_none_no_secret_raised():
"""When the refresh HTTP call fails, _refresh_google_token must return None
silently. It must not raise an exception or surface token/secret details."""
from src.secret_storage import encrypt as _enc
db, Factory = _make_db()
_make_account(db, account_id="acct-fail", owner="dave",
oauth_refresh_token=_enc("ref-tok"))
db.close()
failing_resp = mock.MagicMock()
failing_resp.raise_for_status.side_effect = Exception("401 Unauthorized")
with mock.patch("httpx.post", return_value=failing_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-fail")
assert result is None, "failed refresh must return None, not raise"
def test_refresh_without_credentials_returns_none():
"""_refresh_google_token must return None immediately when the OAuth client
credentials are not configured no DB query, no HTTP call."""
with mock.patch("routes.email_helpers.os.environ.get", return_value=""):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-any")
assert result is None
# ── Password-account regression ───────────────────────────────────
def test_imap_connect_uses_login_for_password_accounts():
"""Existing password-auth IMAP accounts must still call conn.login() and
must NOT trigger the XOAUTH2 authenticate path."""
from routes.email_helpers import _imap_connect
mock_conn = mock.MagicMock()
# _imap_connect calls _get_email_config internally — mock it to return our cfg.
cfg = {
"imap_host": "imap.gmail.com",
"imap_port": 993,
"imap_starttls": False,
"imap_user": "me@gmail.com",
"imap_password": "app-password-xyz",
"oauth_provider": "",
"account_id": "acct-pw",
}
with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
_imap_connect("acct-pw", owner="alice")
mock_conn.login.assert_called_once_with("me@gmail.com", "app-password-xyz")
mock_conn.authenticate.assert_not_called()
def test_imap_connect_uses_xoauth2_for_oauth_accounts():
"""OAuth accounts must call conn.authenticate('XOAUTH2', ...) and must NOT
call conn.login() which would fail without a password."""
from routes.email_helpers import _imap_connect
from src.secret_storage import encrypt as _enc
mock_conn = mock.MagicMock()
future_expiry = str(int(time.time()) + 7200)
cfg = {
"imap_host": "imap.gmail.com",
"imap_port": 993,
"imap_starttls": False,
"imap_user": "me@nyu.edu",
"imap_password": "",
"oauth_provider": "google",
"account_id": "acct-oauth",
"oauth_access_token": _enc("ya29.live_token"),
"oauth_token_expiry": future_expiry,
}
with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
_imap_connect("acct-oauth", owner="alice")
mock_conn.authenticate.assert_called_once()
assert mock_conn.authenticate.call_args[0][0] == "XOAUTH2"
mock_conn.login.assert_not_called()
@pytest.mark.asyncio
async def test_account_list_response_does_not_expose_token_values():
"""The /accounts list route is the client-facing account inventory. It must
expose `oauth_provider` (so the UI can show OAuth status) but never the
access/refresh token values, encrypted or otherwise only boolean
has_*_password flags and the provider name."""
from routes.email_routes import setup_email_routes
from src.secret_storage import encrypt as _enc
raw_access = "ya29.super_secret_access_token"
raw_refresh = "1//super_secret_refresh_token"
db, Factory = _make_db()
_make_account(db, account_id="acct-list", owner="alice",
oauth_provider="google",
oauth_access_token=_enc(raw_access),
oauth_refresh_token=_enc(raw_refresh))
db.close()
router = setup_email_routes()
list_accounts = None
for route in router.routes:
if route.path == "/api/email/accounts" and "GET" in getattr(route, "methods", set()):
list_accounts = route.endpoint
break
assert list_accounts is not None, "accounts list route not found"
with mock.patch("core.database.SessionLocal", Factory):
result = await list_accounts(owner="alice")
blob = json.dumps(result)
assert raw_access not in blob, "raw access token must not appear in list response"
assert raw_refresh not in blob, "raw refresh token must not appear in list response"
assert _enc(raw_access) not in blob, "encrypted token must not be sent to the client either"
acct = result["accounts"][0]
assert acct["oauth_provider"] == "google" # status is exposed
assert "oauth_access_token" not in acct # token value is not
assert "oauth_refresh_token" not in acct
@@ -2,7 +2,14 @@ import os
from pathlib import Path
import pytest
from fastapi import FastAPI
from fastapi import HTTPException
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import NullPool
from core.database import Base, GalleryImage
def _gallery_module():
@@ -53,6 +60,57 @@ def test_gallery_image_path_rejects_symlink_escape(tmp_path, monkeypatch):
assert exc.value.status_code == 400
def test_gallery_replace_rejects_symlink_escape(tmp_path, monkeypatch):
gallery_routes = _gallery_module()
image_dir = tmp_path / "generated_images"
image_dir.mkdir()
outside = tmp_path / "outside.png"
outside.write_bytes(b"outside image root")
link = image_dir / "escape.png"
try:
os.symlink(outside, link)
except (AttributeError, NotImplementedError, OSError) as exc:
pytest.skip(f"symlinks unavailable: {exc}")
engine = create_engine(
f"sqlite:///{tmp_path / 'gallery.db'}",
connect_args={"check_same_thread": False},
poolclass=NullPool,
)
Base.metadata.create_all(engine)
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False)
db = SessionLocal()
try:
db.add(
GalleryImage(
id="img-1",
filename="escape.png",
prompt="escape",
owner="alice",
is_active=True,
)
)
db.commit()
finally:
db.close()
monkeypatch.setattr(gallery_routes, "GALLERY_IMAGE_DIR", image_dir)
monkeypatch.setattr(gallery_routes, "SessionLocal", SessionLocal)
monkeypatch.setattr(gallery_routes, "get_current_user", lambda request: "alice")
app = FastAPI()
app.include_router(gallery_routes.setup_gallery_routes())
client = TestClient(app)
response = client.post(
"/api/gallery/img-1/replace",
files={"image": ("replacement.png", b"replacement bytes", "image/png")},
)
assert response.status_code == 400
assert outside.read_bytes() == b"outside image root"
def test_gallery_file_operations_use_confining_resolver():
source = Path("routes/gallery_routes.py").read_text(encoding="utf-8")
+76 -1
View File
@@ -1286,6 +1286,14 @@ class _ImmediateThread:
self.target()
class _NoopThread:
def __init__(self, target, daemon=None):
self.target = target
def start(self):
return None
def _wait_for(predicate, timeout=2.0):
deadline = time.time() + timeout
while time.time() < deadline:
@@ -1313,6 +1321,7 @@ def _route_ep(
pinned_models=None,
refresh_mode="auto",
refresh_timeout=None,
owner=None,
):
return SimpleNamespace(
id=id,
@@ -1329,7 +1338,7 @@ def _route_ep(
model_refresh_interval=None,
model_refresh_timeout=refresh_timeout,
supports_tools=None,
owner=None,
owner=owner,
created_at=None,
updated_at=None,
)
@@ -1342,6 +1351,72 @@ def _route_request():
)
def test_api_models_rejects_api_token_without_chat_scope(monkeypatch):
router = model_routes.setup_model_routes(model_discovery=None)
def fail_session():
raise AssertionError("model DB should not be queried without chat scope")
monkeypatch.setattr(model_routes, "SessionLocal", fail_session)
request = SimpleNamespace(
state=SimpleNamespace(
current_user="api",
api_token=True,
api_token_owner="alice",
api_token_scopes=["documents:read"],
),
app=SimpleNamespace(
state=SimpleNamespace(
auth_manager=SimpleNamespace(is_configured=True, is_admin=lambda user: False),
),
),
)
with pytest.raises(HTTPException) as exc:
_route_endpoint(router, "/api/models")(request)
assert exc.value.status_code == 403
assert "chat" in str(exc.value.detail)
def test_api_models_scopes_api_token_to_token_owner(monkeypatch):
rows = [
_route_ep("alice", "http://alice.example/v1", cached_models=["alice-model"], owner="alice"),
_route_ep("shared", "http://shared.example/v1", cached_models=["shared-model"], owner=None),
_route_ep("bob", "http://bob.example/v1", cached_models=["bob-model"], owner="bob"),
]
db = _RouteDb(rows)
router = model_routes.setup_model_routes(model_discovery=None)
admin_checks = []
monkeypatch.setattr(model_routes, "ModelEndpoint", _RouteModelEndpoint)
monkeypatch.setattr(model_routes, "SessionLocal", lambda: db)
monkeypatch.setattr(threading, "Thread", _NoopThread)
request = SimpleNamespace(
state=SimpleNamespace(
current_user="api",
api_token=True,
api_token_owner="alice",
api_token_scopes=["chat"],
),
app=SimpleNamespace(
state=SimpleNamespace(
auth_manager=SimpleNamespace(
is_configured=True,
is_admin=lambda user: admin_checks.append(user) or False,
),
),
),
)
result = _route_endpoint(router, "/api/models")(request)
assert [item["endpoint_name"] for item in result["items"]] == ["alice", "shared"]
assert admin_checks == ["alice"]
def test_api_models_returns_cached_proxy_models_without_refresh_probe(monkeypatch):
row = _route_ep(
"proxy",
+188
View File
@@ -0,0 +1,188 @@
"""Owner-scoped note routes must fail closed when the request has no identity.
The notes CRUD routes resolved the acting user with bare get_current_user().
A request that reached them carrying no identity (auth-middleware regression,
SSRF from a sibling service) therefore came through as user=None and the
queries treat None as the single-user mode, i.e. blanket access to every
account's notes: list everything, read/update/delete/pin/archive any row,
reorder globally.
require_user() already encodes the correct policy 401 when auth is
configured, while the documented anonymous modes (AUTH_ENABLED=false,
LOCALHOST_BYPASS on loopback, unconfigured first-run) still pass and
fire-reminder in the same file already used it. The CRUD routes now resolve
the owner through it too.
Test transport note: these drive the ASGI app through ``httpx.ASGITransport``
+ ``httpx.AsyncClient`` rather than ``starlette.testclient.TestClient``.
TestClient runs the app inside a background event-loop thread spun up by
``anyio.from_thread.start_blocking_portal`` and then dispatches each sync
endpoint onto *another* worker thread; on some anyio/httpx/platform
combinations that two-thread handshake deadlocks and ``TestClient(app).get(...)``
simply hangs. ASGITransport runs the whole request on the test's own event
loop no portal thread, no BaseHTTPMiddleware so the suite is portable.
Identity is injected by a pure-ASGI shim that writes the same
``request.state`` fields the real auth middleware sets.
"""
import uuid
from types import SimpleNamespace
import httpx
import pytest
from fastapi import FastAPI
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.pool import NullPool
import core.database as cdb
from core.database import Note
import routes.note_routes as nr
# A deliberately NON-loopback peer. require_user has loopback fall-throughs
# (unconfigured first-run, LOCALHOST_BYPASS); pinning a public-looking client
# keeps every assertion below about the *configured-auth* path and not an
# accidental loopback bypass — the same reason the old fixture leaned on
# TestClient's non-loopback "testclient" host.
_PEER = ("203.0.113.7", 54321)
class _Identity:
"""Pure-ASGI shim mirroring what the auth middleware writes onto
request.state. Pure-ASGI on purpose it stays off Starlette's
BaseHTTPMiddleware + sync-TestClient path, the source of the
``TestClient(app).get(...)`` hang. No x-test-user header => no identity,
the exact state an auth-middleware regression would produce."""
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] == "http":
headers = dict(scope.get("headers") or [])
state = scope.setdefault("state", {})
user = headers.get(b"x-test-user")
if user:
state["current_user"] = user.decode()
if headers.get(b"x-test-api-token"):
state["current_user"] = "api"
state["api_token"] = True
await self.app(scope, receive, send)
def _temp_db(tmp_path):
"""Note routes over a fresh temp DB; returns the session factory."""
engine = create_engine(
f"sqlite:///{tmp_path / 'notes.db'}",
connect_args={"check_same_thread": False},
poolclass=NullPool,
)
cdb.Base.metadata.create_all(engine)
return sessionmaker(bind=engine)
def _build_app(factory, *, configured=True):
app = FastAPI()
app.state.auth_manager = SimpleNamespace(is_configured=configured)
app.include_router(nr.setup_note_routes())
return _Identity(app)
def _client(app):
"""AsyncClient over the ASGI app with a non-loopback peer. Caller drives
it inside ``async with``."""
transport = httpx.ASGITransport(app=app, client=_PEER)
return httpx.AsyncClient(transport=transport, base_url="http://notes.test")
@pytest.fixture
def env(monkeypatch, tmp_path):
"""Configured-auth world: AUTH_ENABLED=true, auth_manager.is_configured,
no LOCALHOST_BYPASS. Identity comes only from the x-test-user header
(mirroring the auth middleware); no header => no identity, the exact state
an auth-middleware regression leaves behind. Seeds one note each for alice
and bob. Returns (app, factory)."""
factory = _temp_db(tmp_path)
monkeypatch.setattr(nr, "SessionLocal", factory)
monkeypatch.setenv("AUTH_ENABLED", "true")
monkeypatch.delenv("LOCALHOST_BYPASS", raising=False)
app = _build_app(factory)
db = factory()
db.add(Note(id="note-alice", owner="alice", title="a", content="x",
items='[{"text": "t", "done": false}]'))
db.add(Note(id="note-bob", owner="bob", title="b", content="y"))
db.commit()
db.close()
return app, factory
async def test_no_identity_fails_closed_on_every_owner_scoped_route(env):
app, _ = env
async with _client(app) as c:
assert (await c.get("/api/notes")).status_code == 401
assert (await c.get("/api/notes/note-alice")).status_code == 401
assert (await c.put("/api/notes/note-alice", json={"title": "pwn"})).status_code == 401
assert (await c.delete("/api/notes/note-alice")).status_code == 401
assert (await c.post("/api/notes/note-alice/pin")).status_code == 401
assert (await c.post("/api/notes/note-alice/archive")).status_code == 401
assert (await c.post("/api/notes/note-alice/items/0/toggle")).status_code == 401
assert (await c.post("/api/notes/reorder", json={"ids": ["note-bob", "note-alice"]})).status_code == 401
assert (await c.post("/api/notes", json={"title": "ghost"})).status_code == 401
async def test_no_identity_did_not_mutate_anything(env):
app, factory = env
async with _client(app) as c:
await c.put("/api/notes/note-alice", json={"title": "pwn"})
await c.post("/api/notes/note-alice/pin")
await c.delete("/api/notes/note-bob")
db = factory()
rows = {n.id: n for n in db.query(Note).all()}
db.close()
assert set(rows) == {"note-alice", "note-bob"}
assert rows["note-alice"].title == "a"
assert not rows["note-alice"].pinned
async def test_authenticated_user_still_scoped_to_own_notes(env):
app, _ = env
alice = {"x-test-user": "alice"}
async with _client(app) as c:
listed = (await c.get("/api/notes", headers=alice)).json()["notes"]
assert [n["id"] for n in listed] == ["note-alice"]
assert (await c.get("/api/notes/note-alice", headers=alice)).status_code == 200
# Someone else's note stays a 404 (don't reveal it exists).
assert (await c.get("/api/notes/note-bob", headers=alice)).status_code == 404
assert (await c.put("/api/notes/note-alice", json={"title": "mine"}, headers=alice)).status_code == 200
async def test_api_token_pseudo_user_is_rejected(env):
"""Bearer tokens must use the scope-aware API routes (require_user's
existing contract), not slip into cookie-session routes as user 'api'."""
app, _ = env
async with _client(app) as c:
r = await c.get("/api/notes", headers={"x-test-api-token": "1"})
assert r.status_code == 403
async def test_auth_disabled_keeps_single_user_mode_working(monkeypatch, tmp_path):
"""AUTH_ENABLED=false is the operator's explicit anonymous mode: no
identity must still mean full single-user access (issue #622 contract),
even with a stale configured auth.json on disk."""
factory = _temp_db(tmp_path)
monkeypatch.setattr(nr, "SessionLocal", factory)
monkeypatch.setenv("AUTH_ENABLED", "false")
app = _build_app(factory)
db = factory()
db.add(Note(id="n1", owner=None, title="solo", content="x"))
db.commit()
db.close()
async with _client(app) as c:
assert [n["id"] for n in (await c.get("/api/notes")).json()["notes"]] == ["n1"]
assert (await c.put("/api/notes/n1", json={"title": "still mine"})).status_code == 200
assert (await c.post("/api/notes/n1/pin")).status_code == 200
@@ -0,0 +1,74 @@
import asyncio
import os
from pathlib import Path
from routes import personal_routes
class _FakePersonalDocs:
def __init__(self):
self.excluded = []
def exclude_file(self, filepath):
self.excluded.append(filepath)
class _FakeRAG:
def __init__(self):
self.deleted_sources = []
def delete_by_source(self, filepath):
self.deleted_sources.append(filepath)
return 1
def _delete_endpoint(personal_docs):
router = personal_routes.setup_personal_routes(personal_docs, None, True)
for route in router.routes:
if getattr(route, "path", "") == "/api/personal/file" and "DELETE" in getattr(route, "methods", set()):
return route.endpoint
raise AssertionError("DELETE /api/personal/file endpoint not found")
def test_delete_file_refuses_symlink_directory_escape(tmp_path, monkeypatch):
uploads = tmp_path / "uploads"
uploads.mkdir()
outside = tmp_path / "outside"
outside.mkdir()
victim = outside / "victim.txt"
victim.write_text("keep me", encoding="utf-8")
os.symlink(outside, uploads / "linked")
docs = _FakePersonalDocs()
rag = _FakeRAG()
monkeypatch.setattr(personal_routes, "UPLOADS_DIR", str(uploads))
monkeypatch.setattr(personal_routes, "get_rag_manager", lambda: rag)
filepath = str(uploads / "linked" / "victim.txt")
result = asyncio.run(_delete_endpoint(docs)(filepath=filepath, owner="alice", _admin=None))
assert result["deleted_from_disk"] is False
assert victim.read_text(encoding="utf-8") == "keep me"
assert docs.excluded == [filepath]
assert rag.deleted_sources == [filepath]
def test_delete_file_removes_regular_file_inside_upload_root(tmp_path, monkeypatch):
uploads = tmp_path / "uploads"
uploads.mkdir()
uploaded_file = uploads / "alice" / "notes.txt"
uploaded_file.parent.mkdir()
uploaded_file.write_text("delete me", encoding="utf-8")
docs = _FakePersonalDocs()
rag = _FakeRAG()
monkeypatch.setattr(personal_routes, "UPLOADS_DIR", str(uploads))
monkeypatch.setattr(personal_routes, "get_rag_manager", lambda: rag)
filepath = str(uploaded_file)
result = asyncio.run(_delete_endpoint(docs)(filepath=filepath, owner="alice", _admin=None))
assert result["deleted_from_disk"] is True
assert not uploaded_file.exists()
assert docs.excluded == [filepath]
assert rag.deleted_sources == [filepath]
+21 -16
View File
@@ -1,16 +1,18 @@
"""Regression guard for issue #1390 — the README banner / ASCII art was not in a
fenced code block, so GitHub's markdown collapsed its leading whitespace and the
box-drawing rules, rendering it misaligned instead of monospace-as-typed.
"""Regression guard for the README title presentation.
This pins that the decorative banner stays inside a ``` code fence.
Originally (#1390) the README opened with an ASCII-art banner that had to live
inside a ``` code fence, otherwise GitHub's markdown collapsed its leading
whitespace and box-drawing rules and rendered it misaligned. The README refresh
(#4306) dropped that banner in favour of a centered wordmark image, so the guard
now pins the wordmark identity instead, while still catching the original failure
mode if an un-fenced ASCII banner is ever reintroduced.
"""
from pathlib import Path
README = Path(__file__).resolve().parent.parent / "README.md"
# Distinctive bits of the banner (box-drawing rule + the kaomoji version line).
# Box-drawing rule from the legacy ASCII banner (the #1390 failure mode).
_RULE = "" * 10
_BANNER_LINE = "Odysseus vers. 1.0"
def _fenced_segments(text: str):
@@ -20,15 +22,18 @@ def _fenced_segments(text: str):
return parts[1::2]
def test_readme_banner_is_inside_a_code_fence():
def test_readme_opens_with_wordmark_title():
# The README must still open with a recognizable Odysseus title: now the
# centered wordmark image rather than an H1 / ASCII banner.
head = "\n".join(README.read_text(encoding="utf-8").splitlines()[:15])
assert 'alt="Odysseus"' in head, "README must open with the Odysseus wordmark image"
def test_reintroduced_ascii_banner_stays_fenced():
# Defensive: if a box-drawing banner is ever added back, it must be fenced so
# GitHub renders it monospace-as-typed (the original #1390 regression).
text = README.read_text(encoding="utf-8")
assert _BANNER_LINE in text, "banner line missing from README"
if _RULE not in text:
return
inside = "\n".join(_fenced_segments(text))
assert _BANNER_LINE in inside, "banner version line must be inside a ``` code fence"
assert _RULE in inside, "banner rule line must be inside a ``` code fence"
def test_readme_title_stays_a_heading():
# The H1 must remain a real heading, not get swallowed into the fence.
first = README.read_text(encoding="utf-8").splitlines()[0]
assert first.strip() == "# Odysseus"
assert _RULE in inside, "ASCII banner rule must be inside a ``` code fence"
+6 -3
View File
@@ -121,9 +121,12 @@ def test_docker_compose_binds_web_ui_to_loopback_by_default():
def test_readme_native_quickstart_uses_loopback():
readme = Path("README.md").read_text(encoding="utf-8")
assert "python -m uvicorn app:app --host 127.0.0.1 --port 7000" in readme
assert "0.0.0.0` only when you intentionally want" in readme
# The README refresh (#4306) moved the native quickstart into docs/setup.md,
# so accept the loopback guidance from either the README or the setup guide.
docs = Path("README.md").read_text(encoding="utf-8")
docs += "\n" + Path("docs/setup.md").read_text(encoding="utf-8")
assert "python -m uvicorn app:app --host 127.0.0.1 --port 7000" in docs
assert "0.0.0.0` only when you intentionally want" in docs
def test_ollama_cookbook_runner_does_not_force_public_bind():