Salastil/odysseus
1
0
Fork 0
mirror of https://github.com/pewdiepie-archdaemon/odysseus.git synced 2026-06-15 17:25:26 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(email): add Google OAuth2 for Google Workspace / .edu IMAP & SMTP (#237)

Browse Source 
* feat(email): add Google OAuth2 for Google Workspace / .edu IMAP & SMTP

Google deprecated basic-auth (password) access for Google Workspace
accounts in May 2025. This means any .edu or org Google email account
could no longer connect via IMAP/SMTP with a username + password —
the email feature was silently broken for a large class of users.

This PR adds full OAuth2 (XOAUTH2) support for Google accounts so
Workspace / .edu emails work out of the box.

## What changed

### Backend
- `core/database.py`: add `oauth_provider`, `oauth_access_token`,
  `oauth_refresh_token`, `oauth_token_expiry`, and `display_name`
  columns to `EmailAccount` + idempotent migration
- `routes/email_helpers.py`: XOAUTH2 auth in `_imap_connect()` and
  `_send_smtp_message()`, automatic token refresh, OAuth fields in
  `_get_email_config()`
- `routes/email_routes.py`: OAuth authorize + callback routes,
  `_smtp_ready()` fix, OAuth fields through `_deliver()` closure,
  `display_name` in `From:` header

### Frontend
- `static/js/settings.js`: "Google Workspace / .edu" provider preset,
  "Connect with Google" button, success/error banner, display name field
- `static/js/document.js`: `_accountCanSend()` recognises OAuth accounts
  as SMTP-capable

* security: sign OAuth state, scope callback by owner, fix quotes & logs

Addresses reviewer feedback on the email OAuth2 PR:

- OAuth state is now HMAC-SHA256 signed (keyed with the app secret from
  secret_storage) encoding account_id + owner + a random nonce, and is
  verified with constant-time comparison in the callback before any
  token write. Replaces the bare account_id state, closing the CSRF /
  state-guessing gap.
- Callback extracts the owner from the verified state and re-checks it
  against EmailAccount.owner before writing tokens, matching the
  ownership guards used elsewhere in the email routes. Single-user mode
  (owner == "") still accepts any account, consistent with
  _assert_owns_account.
- Replaced curly/smart quotes in the Name/Email/Display Name input rows
  with plain ASCII so getElementById lookups and event wiring work.
- Stripped account name, SMTP host/user, owner, and raw provider error
  text from send-config and OAuth logs; failures now surface as generic
  error codes in the redirect instead of raw exception strings.

* test(email): add OAuth2 state, _smtp_ready, and XOAUTH2 tests

Move the OAuth state sign/verify helpers out of the setup_email_routes
closure into module-level make_oauth_state/verify_oauth_state in
email_helpers.py so they can be unit-tested, then add tests/test_email_oauth.py:

- signed state round-trips account_id + owner, nonce is unique per call
- tampered account_id, forged signature, and garbage states are rejected
- _smtp_ready treats an OAuth account (no password) as send-capable, and
  still rejects host+user-only accounts with neither password nor OAuth
- _xoauth2_string / _xoauth2_bytes produce the correct SASL XOAUTH2 framing

14 new tests; existing test_security_regressions.py still passes (28).

* refactor(email): single XOAUTH2 frame helper, use RuntimeError

Polish from self-review before merge:

- Collapse the XOAUTH2 framing to one source of truth: _xoauth2_raw()
  returns the unencoded SASL string used by both the SMTP and IMAP auth
  callbacks (each library base64-encodes it), and _xoauth2_bytes() is
  just its .encode(). Removes the unused base64 _xoauth2_string helper
  and the duplicated inline frame in _send_smtp_message.
- Raise RuntimeError (not bare Exception) for the "OAuth token
  unavailable" path, matching the convention used across src/.
- Update tests accordingly.

All 14 OAuth tests + 28 security regressions pass; SMTP/IMAP XOAUTH2
verified live against a real Workspace account.

* tests(email-oauth): cover the security-sensitive OAuth paths before merge

The previous tests only exercised pure helpers (state signing, _smtp_ready,
XOAUTH2 framing). This adds coverage for the actual token-custody and
ownership behaviour, pinning the real route handlers rather than
re-implementations of their logic.

Real OAuth callback route (pulled live from setup_email_routes()):
- missing code -> generic missing_code redirect, no account id / owner in URL
- provider error -> generic google_error redirect, raw error not echoed
- tampered/invalid state -> invalid_state redirect, auth code never leaked
- signed state with owner mismatch -> token write refused (ownership_error),
  DB row left untouched
- signed state with matching owner -> tokens written encrypted, and only to
  the intended account (a second account stays untouched)

Real accounts-list route:
- exposes oauth_provider status but never the access/refresh token values,
  encrypted or otherwise

Token storage / refresh helpers (isolated in-memory SQLite, mocked HTTP):
- refreshed access token stored encrypted; expiry is a timestamp, not a token
- fresh token uses cache (no refresh call); expired token triggers refresh
- refresh HTTP failure returns None silently, no exception or secret surfaced
- missing client credentials short-circuits to None

Password-account regression:
- password IMAP accounts call conn.login(); OAuth accounts call XOAUTH2
  authenticate() and never login()

28 tests pass (14 prior + 14 new).

* fix(email-oauth): drop raw exception text from token-refresh log

Google token refresh failures now log the account id only, matching
the conservative logging used elsewhere on the OAuth path — no raw
provider/exception details surfacing in logs.

* fix(email-oauth): bring OAuth UI parity to the Integrations email form

The Google Workspace / .edu provider preset, Display Name field, and
Connect-with-Google flow were only wired into the Email-tab account
form. The Integrations-tab form (a separate code path for the same
account type) was missing all three, so the OAuth option was invisible
from that entry point. Mirrors the same PROVIDERS entry, OAuth section,
and connect handler so both forms behave identically.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
This commit is contained in:
Hriday Ranka
2026-06-15 12:02:58 -04:00
committed by GitHub
parent 0750486654
commit 270b8570fc
6 changed files with 1013 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/database.py
+27
View File
@@ -324,6 +324,13 @@ class EmailAccount(TimestampMixin, Base):
smtp_password = Column(String, default="")
from_address = Column(String, default="")
display_name = Column(String, nullable=True) # "Hriday Ranka" — used in From: header
# OAuth2 (Google / Google Workspace). Tokens stored encrypted via secret_storage.
oauth_provider = Column(String, nullable=True) # "google" or None
oauth_access_token = Column(String, nullable=True) # encrypted
oauth_refresh_token = Column(String, nullable=True) # encrypted
oauth_token_expiry = Column(String, nullable=True) # unix timestamp string
__table_args__ = (
Index('ix_email_accounts_owner_default', 'owner', 'is_default'),
@@ -1427,6 +1434,25 @@ def _migrate_add_task_automation_columns():
except Exception as e:
logging.getLogger(__name__).warning(f"task automation migration: {e}")
def _migrate_add_email_oauth_columns():
"""Add Google OAuth and display_name columns to email_accounts if missing."""
try:
with engine.connect() as conn:
cols = [r[1] for r in conn.execute(text("PRAGMA table_info(email_accounts)"))]
for col, typedef in [
("oauth_provider", "TEXT"),
("oauth_access_token", "TEXT"),
("oauth_refresh_token", "TEXT"),
("oauth_token_expiry", "TEXT"),
("display_name", "TEXT"),
]:
if col not in cols:
conn.execute(text(f"ALTER TABLE email_accounts ADD COLUMN {col} {typedef}"))
conn.commit()
except Exception as e:
logging.getLogger(__name__).warning(f"email oauth columns migration: {e}")
def _migrate_add_oauth_config():
"""Add oauth_config column to mcp_servers table if missing."""
try:
@@ -1771,6 +1797,7 @@ def init_db():
_migrate_add_tidy_verdict()
_migrate_add_doc_source_email_cols()
_migrate_add_oauth_config()
_migrate_add_email_oauth_columns()
_migrate_add_task_automation_columns()
_migrate_add_disabled_tools()
_migrate_add_mcp_oauth_tokens_column()
routes/email_helpers.py
+134 -10
View File
@@ -13,6 +13,8 @@ and `email_pollers.py` (the background loops):
"""
import os
import base64
import time
import imaplib
import smtplib
import email as email_mod
@@ -38,6 +40,106 @@ from src.secret_storage import decrypt as _decrypt
logger = logging.getLogger(__name__)
def _xoauth2_raw(user: str, access_token: str) -> str:
"""The SASL XOAUTH2 initial-response string (unencoded).
Both smtplib.SMTP.auth() and imaplib.IMAP4.authenticate() base64-encode
the value their callback returns, so callers pass this raw form — never
pre-encoded — to avoid double base64.
"""
return f"user={user}\x01auth=Bearer {access_token}\x01\x01"
def _xoauth2_bytes(user: str, access_token: str) -> bytes:
"""Raw XOAUTH2 bytes for imaplib's authenticate() callback."""
return _xoauth2_raw(user, access_token).encode()
def make_oauth_state(account_id: str, owner: str) -> str:
"""Return an HMAC-signed, base64-encoded OAuth state token.
Encodes account_id + owner + a random nonce, signed with the app secret
so the callback can validate that the flow was initiated by an
authenticated, owning user (CSRF / state-forgery protection).
"""
import hmac as _hmac, hashlib as _hl, secrets as _sec
from src.secret_storage import _load_or_create_key
nonce = _sec.token_hex(16)
payload = json.dumps({"a": account_id, "o": owner, "n": nonce}, separators=(",", ":"))
sig = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
return base64.urlsafe_b64encode(f"{payload}|{sig}".encode()).decode()
def verify_oauth_state(state: str) -> dict | None:
"""Verify an OAuth state token's HMAC signature.
Returns the decoded payload dict ({"a", "o", "n"}) on success, or None if
the token is malformed, tampered, or signed with a different key.
"""
import hmac as _hmac, hashlib as _hl
from src.secret_storage import _load_or_create_key
try:
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload, sig = decoded.rsplit("|", 1)
expected = _hmac.new(_load_or_create_key(), payload.encode(), _hl.sha256).hexdigest()
if not _hmac.compare_digest(sig, expected):
return None
return json.loads(payload)
except Exception:
return None
def _refresh_google_token(account_id: str) -> str | None:
"""Exchange the stored refresh token for a new access token and persist it."""
import httpx
from core.database import SessionLocal as _SL, EmailAccount as _EA
from src.secret_storage import encrypt as _enc, decrypt as _dec
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
if not client_id or not client_secret:
return None
db = _SL()
try:
row = db.get(_EA, account_id)
if not row or not row.oauth_refresh_token:
return None
refresh_token = _dec(row.oauth_refresh_token or "")
if not refresh_token:
return None
resp = httpx.post("https://oauth2.googleapis.com/token", data={
"client_id": client_id,
"client_secret": client_secret,
"refresh_token": refresh_token,
"grant_type": "refresh_token",
}, timeout=10)
resp.raise_for_status()
data = resp.json()
access_token = data["access_token"]
row.oauth_access_token = _enc(access_token)
row.oauth_token_expiry = str(int(time.time()) + data.get("expires_in", 3600))
db.commit()
return access_token
except Exception:
logger.warning(f"Google token refresh failed for account {account_id}")
return None
finally:
db.close()
def _get_valid_google_token(account_id: str, cfg: dict) -> str | None:
"""Return a valid Google access token, refreshing if expired or missing."""
from src.secret_storage import decrypt as _dec
access_token = _dec(cfg.get("oauth_access_token") or "")
expiry_str = cfg.get("oauth_token_expiry") or ""
if access_token and expiry_str:
try:
if int(expiry_str) - 60 > time.time():
return access_token
except (ValueError, TypeError):
pass
return _refresh_google_token(account_id)
def _smtp_security_mode(cfg: dict) -> str:
raw = str(cfg.get("smtp_security") or "").strip().lower()
if raw in {"ssl", "starttls", "none"}:
@@ -54,20 +156,29 @@ def _send_smtp_message(cfg: dict, from_addr: str, recipients: list[str], message
port = int(cfg.get("smtp_port") or 465)
user = cfg.get("smtp_user") or ""
password = cfg.get("smtp_password") or ""
def _auth_smtp(smtp):
if cfg.get("oauth_provider") == "google":
token = _get_valid_google_token(cfg.get("account_id"), cfg)
if not token:
raise RuntimeError("Google OAuth token unavailable — reconnect the account")
smtp.ehlo()
smtp.auth("XOAUTH2", lambda challenge=None: _xoauth2_raw(user, token), initial_response_ok=True)
elif user and password:
smtp.login(user, password)
security = _smtp_security_mode(cfg)
if security == "ssl":
with smtplib.SMTP_SSL(host, port, timeout=timeout) as smtp:
if user and password:
smtp.login(user, password)
_auth_smtp(smtp)
smtp.sendmail(from_addr, recipients, message)
return
with smtplib.SMTP(host, port, timeout=timeout) as smtp:
if security == "starttls":
smtp.starttls()
if user and password:
smtp.login(user, password)
_auth_smtp(smtp)
smtp.sendmail(from_addr, recipients, message)
@@ -701,10 +812,16 @@ def _get_email_config(account_id: str | None = None, owner: str = "") -> dict:
"imap_password": _decrypt(row.imap_password or ""),
"imap_starttls": bool(row.imap_starttls),
"from_address": row.from_address or row.imap_user or "",
"oauth_provider": row.oauth_provider or "",
"oauth_access_token": row.oauth_access_token or "",
"oauth_refresh_token": row.oauth_refresh_token or "",
"oauth_token_expiry": row.oauth_token_expiry or "",
"display_name": row.display_name or "",
}
if not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
is_oauth = bool(cfg.get("oauth_provider"))
if not is_oauth and not (cfg["smtp_host"] and cfg["smtp_user"] and cfg["smtp_password"]):
logger.warning(f"SMTP not configured for account {row.name!r}")
if not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
if not is_oauth and not (cfg["imap_host"] and cfg["imap_user"] and cfg["imap_password"]):
logger.warning(f"IMAP not configured for account {row.name!r}")
return cfg
finally:
@@ -825,12 +942,19 @@ def _imap_connect(account_id: str | None = None, owner: str = "",
timeout=timeout,
)
try:
conn.login(cfg["imap_user"], cfg["imap_password"])
if cfg.get("oauth_provider") == "google":
token = _get_valid_google_token(cfg.get("account_id"), cfg)
if not token:
raise RuntimeError("Google OAuth token unavailable — reconnect the account in Settings → Integrations")
conn.authenticate("XOAUTH2", lambda x: _xoauth2_bytes(cfg["imap_user"], token))
else:
conn.login(cfg["imap_user"], cfg["imap_password"])
except Exception:
# A failed AUTHENTICATE (e.g. an Office 365 app password on an
# MFA-enabled tenant, #3174) otherwise orphans the already-connected
# socket; close it before propagating so a misconfigured account
# can't leak one descriptor per retry / background poller pass.
# MFA-enabled tenant, #3174, or an expired/revoked OAuth token)
# otherwise orphans the already-connected socket; close it before
# propagating so a misconfigured account can't leak one descriptor
# per retry / background poller pass.
try:
conn.shutdown()
except Exception:
routes/email_routes.py
+142 -5
View File
@@ -13,7 +13,9 @@ handlers need. The split is mechanical — no behavior change.
"""
import asyncio
import os
import sqlite3 as _sql3
import time
import email as email_mod
import email.header
import email.utils
@@ -43,6 +45,7 @@ from routes.email_helpers import (
_load_settings, _save_settings, _get_email_config,
_send_smtp_message, _smtp_security_mode,
_IMAP_TIMEOUT_SECONDS, _open_imap_connection,
make_oauth_state, verify_oauth_state,
_imap_connect, _imap, _decode_header, _detect_sent_folder, _detect_drafts_folder,
_extract_attachment_text, _list_attachments_from_msg,
_extract_attachment_to_disk, _extract_html, _extract_text,
@@ -285,7 +288,9 @@ def _group_uid_fetch_records(msg_data) -> list:
def _smtp_ready(cfg: dict) -> bool:
return bool(cfg.get("smtp_host") and cfg.get("smtp_user") and cfg.get("smtp_password"))
if not cfg.get("smtp_host") or not cfg.get("smtp_user"):
return False
return bool(cfg.get("smtp_password") or cfg.get("oauth_provider"))
def _resolve_send_config(account_id: str | None = None, owner: str = "") -> dict:
@@ -2021,7 +2026,7 @@ def setup_email_routes():
outer = MIMEMultipart("alternative")
body_container = outer
outer["From"] = cfg["from_address"]
outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
outer["To"] = to
if cc:
outer["Cc"] = cc
@@ -2285,6 +2290,7 @@ def setup_email_routes():
try:
cfg = _resolve_send_config(req.account_id, owner=owner)
except Exception as e:
logger.warning(f"No SMTP-capable account resolved: {e}")
return {"success": False, "error": str(e) or "No SMTP-capable email account configured"}
# Use 'mixed' if we have attachments, 'alternative' otherwise
@@ -2297,7 +2303,7 @@ def setup_email_routes():
outer = MIMEMultipart("alternative")
body_container = outer
outer["From"] = cfg["from_address"]
outer["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
outer["To"] = req.to
if req.cc:
outer["Cc"] = req.cc
@@ -2348,6 +2354,10 @@ def setup_email_routes():
_account_id = cfg.get("account_id") or req.account_id # capture for the IMAP append in the closure
_in_reply_to = (req.in_reply_to or "").strip()
_oauth_provider = cfg.get("oauth_provider") or ""
_oauth_access_token = cfg.get("oauth_access_token") or ""
_oauth_refresh_token = cfg.get("oauth_refresh_token") or ""
_oauth_token_expiry = cfg.get("oauth_token_expiry") or ""
def _deliver():
try:
@@ -2358,6 +2368,11 @@ def setup_email_routes():
"smtp_security": _smtp_security,
"smtp_user": _smtp_user,
"smtp_password": _smtp_pw,
"account_id": _account_id,
"oauth_provider": _oauth_provider,
"oauth_access_token": _oauth_access_token,
"oauth_refresh_token": _oauth_refresh_token,
"oauth_token_expiry": _oauth_token_expiry,
},
_from,
_recipients,
@@ -2470,7 +2485,7 @@ def setup_email_routes():
msg.attach(MIMEText(_draft_html, "html", "utf-8"))
else:
msg = MIMEText(req.body, "plain", "utf-8")
msg["From"] = cfg["from_address"]
msg["From"] = email.utils.formataddr((cfg.get("display_name") or "", cfg["from_address"]))
msg["To"] = req.to
if req.cc:
msg["Cc"] = req.cc
@@ -3122,6 +3137,8 @@ def setup_email_routes():
"from_address": r.from_address or "",
"has_imap_password": bool(r.imap_password),
"has_smtp_password": bool(r.smtp_password),
"oauth_provider": r.oauth_provider or "",
"display_name": r.display_name or "",
})
return {"accounts": out}
finally:
@@ -3154,6 +3171,7 @@ def setup_email_routes():
smtp_user=(data.get("smtp_user") or "").strip(),
smtp_password=_enc(data.get("smtp_password") or ""),
from_address=(data.get("from_address") or "").strip(),
display_name=(data.get("display_name") or "").strip(),
# SECURITY: stamp the creator so all subsequent reads / mutations
# can filter by user. Without this every new account leaks to
# every other user.
@@ -3188,7 +3206,7 @@ def setup_email_routes():
if not row:
return {"ok": False, "error": "Account not found"}
# Simple fields
for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address"):
for key in ("name", "imap_host", "imap_user", "smtp_host", "smtp_user", "from_address", "display_name"):
if key in data:
setattr(row, key, (data[key] or "").strip())
for key in ("imap_port", "smtp_port"):
@@ -3377,4 +3395,123 @@ def setup_email_routes():
finally:
db.close()
# ── Google OAuth2 routes ──
@router.get("/oauth/google/authorize")
async def google_oauth_authorize(account_id: str = Query(...), request: Request = None, owner: str = Depends(require_user)):
import urllib.parse
_assert_owns_account(account_id, owner)
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
if not client_id:
raise HTTPException(400, "GOOGLE_OAUTH_CLIENT_ID not set — add it to .env")
redirect_uri = (
os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
)
state = make_oauth_state(account_id, owner)
params = urllib.parse.urlencode({
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": "https://mail.google.com/ email",
"access_type": "offline",
"prompt": "consent",
"state": state,
})
from fastapi.responses import RedirectResponse as _RR
return _RR(f"https://accounts.google.com/o/oauth2/v2/auth?{params}")
@router.get("/oauth/google/callback")
async def google_oauth_callback(
code: str = Query(None),
state: str = Query(None),
error: str = Query(None),
request: Request = None,
):
import urllib.parse
from fastapi.responses import RedirectResponse as _RR
if error:
return _RR("/?section=integrations&email_oauth_error=google_error")
if not code or not state:
return _RR("/?section=integrations&email_oauth_error=missing_code")
state_data = verify_oauth_state(state)
if not state_data:
return _RR("/?section=integrations&email_oauth_error=invalid_state")
account_id = state_data.get("a", "")
owner = state_data.get("o", "")
client_id = os.environ.get("GOOGLE_OAUTH_CLIENT_ID", "")
client_secret = os.environ.get("GOOGLE_OAUTH_CLIENT_SECRET", "")
redirect_uri = (
os.environ.get("GOOGLE_OAUTH_REDIRECT_URI")
or f"http://{request.headers.get('host', 'localhost:7000')}/api/email/oauth/google/callback"
)
import httpx as _httpx
try:
resp = _httpx.post("https://oauth2.googleapis.com/token", data={
"code": code,
"client_id": client_id,
"client_secret": client_secret,
"redirect_uri": redirect_uri,
"grant_type": "authorization_code",
}, timeout=10)
resp.raise_for_status()
data = resp.json()
except Exception:
logger.warning("Google token exchange failed")
return _RR("/?section=integrations&email_oauth_error=token_exchange_failed")
access_token = data.get("access_token", "")
refresh_token = data.get("refresh_token", "")
expiry = str(int(time.time()) + data.get("expires_in", 3600))
# Fetch the email address from userinfo so we can auto-fill imap_user.
email_addr = ""
display_name = ""
try:
ui = _httpx.get("https://www.googleapis.com/oauth2/v1/userinfo",
headers={"Authorization": f"Bearer {access_token}"}, timeout=10)
if ui.is_success:
ui_data = ui.json()
email_addr = ui_data.get("email", "")
display_name = ui_data.get("name", "")
except Exception:
pass
from core.database import SessionLocal, EmailAccount
from src.secret_storage import encrypt as _enc
db = SessionLocal()
try:
row = db.query(EmailAccount).filter(EmailAccount.id == account_id).first()
if not row:
return _RR("/?section=integrations&email_oauth_error=account_not_found")
# SECURITY: verify the account belongs to the initiating user.
if owner and row.owner and row.owner != owner:
logger.warning("OAuth callback owner mismatch — rejecting token write")
return _RR("/?section=integrations&email_oauth_error=ownership_error")
row.oauth_provider = "google"
row.oauth_access_token = _enc(access_token)
if refresh_token:
row.oauth_refresh_token = _enc(refresh_token)
row.oauth_token_expiry = expiry
# Auto-fill Google IMAP/SMTP settings if not already configured.
if not row.imap_host:
row.imap_host = "imap.gmail.com"
row.imap_port = 993
row.imap_starttls = False
if not row.smtp_host:
row.smtp_host = "smtp.gmail.com"
row.smtp_port = 587
if email_addr:
if not row.imap_user:
row.imap_user = email_addr
if not row.smtp_user:
row.smtp_user = email_addr
if not row.from_address:
row.from_address = email_addr
if not row.name or row.name == row.id:
row.name = email_addr
if display_name and not row.display_name:
row.display_name = display_name
db.commit()
finally:
db.close()
return _RR("/?section=integrations&email_oauth_success=1")
return router
static/js/document.js
+2 -1
View File
@@ -87,7 +87,8 @@ import * as Modals from './modalManager.js';
}
function _accountCanSend(account) {
return !!(account && account.smtp_host && account.smtp_user && account.has_smtp_password);
if (!account || !account.smtp_host || !account.smtp_user) return false;
return !!(account.has_smtp_password || account.oauth_provider);
}
async function _resolveComposeSendAccountId() {
static/js/settings.js
+128 -10
View File
@@ -2913,13 +2913,14 @@ async function initEmailAccountsSettings() {
// IMAP and SMTP. Dovecot is IMAP-only here; the host is intentionally
// blank because it may live on another machine (DNS, LAN, Tailscale).
const PROVIDERS = {
gmail: { label: 'Gmail', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
migadu: { label: 'Migadu', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
fastmail: { label: 'Fastmail', imap: { host: 'imap.fastmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com', port: 465 } },
yahoo: { label: 'Yahoo', imap: { host: 'imap.mail.yahoo.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com', port: 465 } },
dovecot: { label: 'Dovecot IMAP (no SMTP)', imap: { host: '', port: 31143, starttls: false }, smtp: { host: '', port: 465 } },
gmail: { label: 'Gmail', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
google_workspace: { label: 'Google Workspace / .edu', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 587 }, oauth: 'google' },
migadu: { label: 'Migadu', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
fastmail: { label: 'Fastmail', imap: { host: 'imap.fastmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.fastmail.com', port: 465 } },
yahoo: { label: 'Yahoo', imap: { host: 'imap.mail.yahoo.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.yahoo.com', port: 465 } },
dovecot: { label: 'Dovecot IMAP (no SMTP)', imap: { host: '', port: 31143, starttls: false }, smtp: { host: '', port: 465 } },
};
const _providerOptions = Object.entries(PROVIDERS)
.map(([k, v]) => `<option value="${k}">${esc(v.label)}</option>`)
@@ -2932,11 +2933,17 @@ async function initEmailAccountsSettings() {
<div id="eaf-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
<div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="eaf-name" class="settings-input" placeholder="(optional — leave blank to use email)" value="${esc(a.name || '')}"></div>
<div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="eaf-from" class="settings-input" placeholder="you@example.com" value="${esc(a.from_address || '')}"></div>
<div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="eaf-display-name" class="settings-input" placeholder="Your Name" value="${esc(a.display_name || '')}"></div>
<div id="eaf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
<div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 — required for Workspace / .edu accounts</div>
<div id="eaf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${a.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
<button type="button" id="eaf-oauth-btn" class="admin-btn-add" style="font-size:11px">${a.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
</div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:6px 0 2px">IMAP (Receiving)</div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="eaf-imap-host" class="settings-input" value="${esc(a.imap_host || '')}"></div>
<div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="eaf-imap-port" class="settings-input" type="number" value="${esc(a.imap_port || 993)}" style="max-width:100px"></div>
<div class="settings-row"><label class="settings-label">Username${_hint('Usually your full email address.')}</label><input id="eaf-imap-user" class="settings-input" value="${esc(a.imap_user || '')}"></div>
<div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div>
<div class="eaf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('Your IMAP login password. Use an app-specific password if your provider requires 2FA. Outlook / Office 365 generally requires OAuth and will not work with a normal password here.')}</label><input id="eaf-imap-pass" class="settings-input" type="password" placeholder="${isEdit && a.has_imap_password ? '(unchanged)' : ''}"></div></div>
<div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch"><input type="checkbox" id="eaf-imap-starttls" ${a.imap_starttls !== false ? 'checked' : ''}><span class="admin-slider"></span></label></div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px">SMTP (Sending) <span style="font-weight:normal;opacity:0.7">— optional, leave blank for read-only</span></div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com, smtp.migadu.com. Leave blank to make this account read-only.')}</label><input id="eaf-smtp-host" class="settings-input" value="${esc(a.smtp_host || '')}"></div>
@@ -2959,6 +2966,16 @@ async function initEmailAccountsSettings() {
</div>
`;
// Show/hide OAuth section and password fields based on provider selection.
function _syncOauthUI(providerKey) {
const p = PROVIDERS[providerKey];
const isOauth = !!(p && p.oauth);
el('eaf-oauth-section').style.display = isOauth ? '' : 'none';
formEl.querySelectorAll('.eaf-password-section').forEach(r => {
r.style.display = isOauth ? 'none' : '';
});
}
const eafProviderNotes = {
outlook: {
title: 'Outlook / Office 365 needs OAuth',
@@ -2983,13 +3000,41 @@ async function initEmailAccountsSettings() {
el('eaf-provider').addEventListener('change', (e) => {
_renderEafProviderNote(e.target.value);
const p = PROVIDERS[e.target.value];
if (!p) return;
if (!p) { _syncOauthUI(''); return; }
el('eaf-imap-host').value = p.imap.host;
el('eaf-imap-port').value = p.imap.port;
el('eaf-imap-starttls').checked = !!p.imap.starttls;
el('eaf-smtp-host').value = p.smtp.host;
el('eaf-smtp-port').value = p.smtp.port;
el('eaf-smtp-security').value = p.smtp.security || ((parseInt(p.smtp.port || 465) === 587) ? 'starttls' : 'ssl');
_syncOauthUI(e.target.value);
});
// Init OAuth UI for accounts already connected via OAuth.
if (a.oauth_provider === 'google') _syncOauthUI('google_workspace');
// "Connect with Google" button — save the account first, then redirect to OAuth.
el('eaf-oauth-btn').addEventListener('click', async () => {
// Must save the account first to get an account_id to pass to the OAuth flow.
const body = {
name: el('eaf-name').value.trim() || el('eaf-from').value.trim(),
from_address: el('eaf-from').value.trim(),
imap_host: el('eaf-imap-host').value.trim(),
imap_port: parseInt(el('eaf-imap-port').value) || 993,
imap_user: el('eaf-imap-user').value.trim(),
imap_starttls: el('eaf-imap-starttls').checked,
smtp_host: el('eaf-smtp-host').value.trim(),
smtp_port: parseInt(el('eaf-smtp-port').value) || 587,
smtp_user: el('eaf-imap-user').value.trim(),
};
if (!body.name) { el('eaf-msg').textContent = 'Enter a Name or Email first'; el('eaf-msg').style.color = 'var(--red)'; return; }
const url = isEdit ? `/api/email/accounts/${a.id}` : '/api/email/accounts';
const method = isEdit ? 'PUT' : 'POST';
const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
const d = await r.json();
if (!d.ok) { el('eaf-msg').textContent = d.error || 'Save failed'; el('eaf-msg').style.color = 'var(--red)'; return; }
const accId = isEdit ? a.id : d.id;
window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
});
el('eaf-smtp-security').value = _smtpSecurity(a);
@@ -3009,6 +3054,7 @@ async function initEmailAccountsSettings() {
const body = {
name: el('eaf-name').value.trim(),
from_address: el('eaf-from').value.trim(),
display_name: el('eaf-display-name').value.trim(),
imap_host: el('eaf-imap-host').value.trim(),
imap_port: parseInt(el('eaf-imap-port').value) || 993,
imap_user: el('eaf-imap-user').value.trim(),
@@ -4317,6 +4363,7 @@ async function initUnifiedIntegrations() {
// it may be remote (DNS, LAN, Tailscale), not localhost.
const PROVIDERS = {
gmail: { label: 'Gmail', emailEx: 'you@gmail.com', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 465 } },
google_workspace: { label: 'Google Workspace / .edu', emailEx: 'you@yourschool.edu', imap: { host: 'imap.gmail.com', port: 993, starttls: false }, smtp: { host: 'smtp.gmail.com', port: 587 }, oauth: 'google' },
migadu: { label: 'Migadu', emailEx: 'you@yourdomain.com', imap: { host: 'imap.migadu.com', port: 993, starttls: false }, smtp: { host: 'smtp.migadu.com', port: 465 } },
icloud: { label: 'iCloud', emailEx: 'you@icloud.com', imap: { host: 'imap.mail.me.com', port: 993, starttls: false }, smtp: { host: 'smtp.mail.me.com', port: 587 } },
outlook: { label: 'Outlook / Office 365', emailEx: 'you@outlook.com', imap: { host: 'outlook.office365.com', port: 993, starttls: false }, smtp: { host: 'smtp.office365.com', port: 587 } },
@@ -4334,6 +4381,7 @@ async function initUnifiedIntegrations() {
const PROV_LOGO = {
'': _customLogo,
gmail: _letterLogo('G', '#ea4335'),
google_workspace: _letterLogo('G', '#ea4335'),
migadu: _letterLogo('M', '#3aa39d'),
icloud: _letterLogo('i', '#3693f3'),
outlook: _letterLogo('O', '#0078d4'),
@@ -4362,11 +4410,17 @@ async function initUnifiedIntegrations() {
<div id="uf-email-provider-note" style="display:none;font-size:11px;line-height:1.5;padding:8px 10px;margin:2px 0 4px;border:1px solid color-mix(in srgb, var(--fg) 15%, transparent);border-left:3px solid var(--accent, var(--red));border-radius:4px;background:color-mix(in srgb, var(--fg) 4%, transparent);"></div>
<div class="settings-row"><label class="settings-label">Name${_hint('Optional label for this account (e.g. “Work” or “Personal”). Leave blank to use the email address.')}</label><input id="uf-email-name" class="settings-input" placeholder="(optional — leave blank to use email)"></div>
<div class="settings-row"><label class="settings-label">Email${_hint('Your email address. Used as the From: header on outgoing mail and as the display label when Name is blank.')}</label><input id="uf-email-from" class="settings-input" placeholder="you@example.com"></div>
<div class="settings-row"><label class="settings-label">Display Name${_hint('Your name as it appears in the From: field of emails you send, e.g. Jane Smith. Auto-filled from Google during OAuth.')}</label><input id="uf-display-name" class="settings-input" placeholder="Your Name"></div>
<div id="uf-oauth-section" style="display:none;margin:8px 0;padding:10px;border:1px solid var(--border);border-radius:6px;background:color-mix(in srgb,var(--accent,#50fa7b) 6%,transparent)">
<div style="font-size:11px;font-weight:600;margin-bottom:6px">Google OAuth2 — required for Workspace / .edu accounts</div>
<div id="uf-oauth-status" style="font-size:11px;opacity:0.7;margin-bottom:6px">${existing && existing.oauth_provider === 'google' ? '✓ Connected via Google OAuth' : 'Not connected — click below to authorize'}</div>
<button type="button" id="uf-oauth-btn" class="admin-btn-add" style="font-size:11px">${existing && existing.oauth_provider === 'google' ? 'Reconnect with Google' : 'Connect with Google'}</button>
</div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:4px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><polyline points="22 12 16 12 14 15 10 15 8 12 2 12"/><path d="M5.45 5.11 2 12v6a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-6l-3.45-6.89A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z"/></svg>IMAP (Receiving)</div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your IMAP server, e.g. imap.gmail.com, imap.migadu.com, a LAN host, or a Tailscale IP for Dovecot.')}</label><input id="uf-imap-host" class="settings-input" placeholder="imap.example.com"></div>
<div class="settings-row"><label class="settings-label">Port${_hint('993 for IMAPS (most providers), 143 for plain or STARTTLS. Local servers often use a custom port like 31143.')}</label><input id="uf-imap-port" class="settings-input" type="number" placeholder="993" style="max-width:100px"></div>
<div class="settings-row"><label class="settings-label">Username${_hint('Yes — your full email address goes here too (e.g. you@gmail.com). Same as the Email field above for almost every provider.')}</label><input id="uf-imap-user" class="settings-input" placeholder="you@example.com"></div>
<div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div>
<div class="uf-password-section"><div class="settings-row"><label class="settings-label">Password${_hint('For Gmail, iCloud, and Yahoo: paste your App Password (NOT your normal account password). For Migadu and Fastmail, your mailbox password usually works. Outlook / Office 365 generally requires OAuth and will not work with this password form.')}</label><input id="uf-imap-pass" class="settings-input" type="password" placeholder="${placeholderPass}"></div></div>
<div class="settings-row"><label class="settings-label">STARTTLS${_hint('Turn ON for port 143/587 to upgrade plain to TLS. Turn OFF for port 993 (IMAPS — already encrypted) or a local server with no TLS configured.')}</label><label class="admin-switch" style="margin-left:0"><input type="checkbox" id="uf-imap-starttls" checked><span class="admin-slider"></span></label></div>
<div style="font-size:11px;font-weight:600;opacity:0.6;margin:8px 0 2px;display:flex;align-items:center;gap:5px;"><svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color:var(--accent, var(--red));flex-shrink:0;" aria-hidden="true"><line x1="22" y1="2" x2="11" y2="13"/><polygon points="22 2 15 22 11 13 2 9 22 2"/></svg>SMTP (Sending) <span style="font-weight:normal;opacity:0.7">— optional, leave blank for read-only</span></div>
<div class="settings-row"><label class="settings-label">Host${_hint('Your outgoing-mail server, e.g. smtp.gmail.com. Leave blank to make this account read-only.')}</label><input id="uf-smtp-host" class="settings-input" placeholder="smtp.example.com"></div>
@@ -4491,6 +4545,16 @@ async function initUnifiedIntegrations() {
</div>`;
};
// Show/hide the OAuth section and password fields based on provider selection.
function _syncOauthUI(providerKey) {
const p = PROVIDERS[providerKey];
const isOauth = !!(p && p.oauth);
el('uf-oauth-section').style.display = isOauth ? '' : 'none';
formEl.querySelectorAll('.uf-password-section').forEach(r => {
r.style.display = isOauth ? 'none' : '';
});
}
// Custom dropdown wire-up — the native <select> stays in the DOM as the
// data source and accessibility target, but the visible UI is a button +
// popup so each provider row can render with its SVG logo. Selecting an
@@ -4547,6 +4611,7 @@ async function initUnifiedIntegrations() {
el('uf-email-provider').addEventListener('change', (e) => {
const key = e.target.value;
_renderProviderNote(key);
_syncOauthUI(key);
const p = PROVIDERS[key];
if (!p) return;
el('uf-imap-host').value = p.imap.host;
@@ -4562,6 +4627,23 @@ async function initUnifiedIntegrations() {
}
});
// Init OAuth UI for accounts already connected via OAuth.
if (existing && existing.oauth_provider === 'google') _syncOauthUI('google_workspace');
// "Connect with Google" — save the account first, then redirect to OAuth.
el('uf-oauth-btn').addEventListener('click', async () => {
const body = _collectBody();
if (!body.name) body.name = body.from_address;
if (!body.name) { el('uf-email-msg').textContent = 'Enter a Name or Email first'; el('uf-email-msg').style.color = 'var(--red)'; return; }
const url = isEdit ? `/api/email/accounts/${editId}` : '/api/email/accounts';
const method = isEdit ? 'PUT' : 'POST';
const r = await fetch(url, { method, credentials: 'same-origin', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body) });
const d = await r.json();
if (!(d.ok || d.id)) { el('uf-email-msg').textContent = d.error || 'Save failed'; el('uf-email-msg').style.color = 'var(--red)'; return; }
const accId = isEdit ? editId : d.id;
window.location.href = `/api/email/oauth/google/authorize?account_id=${encodeURIComponent(accId)}`;
});
// "Same as IMAP" toggle — hide the SMTP creds rows when on.
const _syncSmtpSame = () => {
const same = el('uf-smtp-same').checked;
@@ -4574,6 +4656,7 @@ async function initUnifiedIntegrations() {
if (existing) {
el('uf-email-name').value = existing.name || '';
el('uf-email-from').value = existing.from_address || '';
el('uf-display-name').value = existing.display_name || '';
el('uf-imap-host').value = existing.imap_host || '';
el('uf-imap-port').value = existing.imap_port || 993;
el('uf-imap-user').value = existing.imap_user || '';
@@ -4622,6 +4705,7 @@ async function initUnifiedIntegrations() {
const body = {
name: el('uf-email-name').value.trim(),
from_address: el('uf-email-from').value.trim(),
display_name: el('uf-display-name').value.trim(),
imap_host: el('uf-imap-host').value.trim(),
imap_port: parseInt(el('uf-imap-port').value) || 993,
imap_user: el('uf-imap-user').value.trim(),
@@ -5650,6 +5734,40 @@ export function close() {
}
}
// Handle redirect back from Google OAuth2 — open settings to integrations and show status.
(function _handleOauthRedirect() {
const sp = new URLSearchParams(window.location.search);
if (!sp.has('email_oauth_success') && !sp.has('email_oauth_error')) return;
// Strip params from URL without a page reload.
const clean = window.location.pathname + window.location.hash;
window.history.replaceState(null, '', clean);
const success = sp.has('email_oauth_success');
const errMsg = sp.get('email_oauth_error') || '';
// Open settings → integrations after the app has initialised.
function _tryOpen() {
if (window.settingsModule && typeof window.settingsModule.open === 'function') {
window.settingsModule.open('integrations');
// Brief toast-style banner.
const banner = document.createElement('div');
banner.textContent = success
? '✓ Google account connected — email is ready'
: `Google OAuth failed: ${errMsg || 'unknown error'}`;
Object.assign(banner.style, {
position: 'fixed', bottom: '24px', left: '50%', transform: 'translateX(-50%)',
background: success ? 'var(--accent, #50fa7b)' : 'var(--red, #ff5555)',
color: '#000', padding: '8px 18px', borderRadius: '6px', fontSize: '12px',
fontWeight: '600', zIndex: '99999', pointerEvents: 'none',
boxShadow: '0 2px 12px rgba(0,0,0,0.3)',
});
document.body.appendChild(banner);
setTimeout(() => banner.remove(), 4000);
} else {
setTimeout(_tryOpen, 100);
}
}
_tryOpen();
})();
const settingsModule = { open, close, initIntegrations, initUnifiedIntegrations, syncAdminVisibility, refreshAiModelEndpoints };
tests/test_email_oauth.py
+580
View File
@@ -0,0 +1,580 @@
"""Tests for the Google OAuth2 email helpers.
Covers the security-critical surface added for Google Workspace / .edu
IMAP/SMTP support:
- `make_oauth_state` / `verify_oauth_state` — HMAC-signed OAuth state so the
callback can't be CSRF'd or have its account_id/owner tampered with.
- `_smtp_ready` — an OAuth account (no stored password) must still count as
send-capable; a host+user-only account without password or OAuth must not.
- `_xoauth2_raw` / `_xoauth2_bytes` — SASL XOAUTH2 framing for SMTP/IMAP.
- `_refresh_google_token` — token refresh stores result encrypted; failure is
silent (no token/secret in logs or return value).
- `_get_valid_google_token` — uses cached token when fresh; calls refresh when
expired.
- `google_oauth_callback` (real route) — invalid/tampered/missing state and
provider errors return generic redirects with no PII; owner mismatch refuses
the token write; a valid owner writes encrypted tokens only to the intended
account.
- `list_email_accounts` (real route) — exposes OAuth status but never token
values.
- `_imap_connect` — password accounts use login(); OAuth accounts use XOAUTH2.
Route tests pull the live endpoint out of `setup_email_routes()` and call it
directly — they pin the real handler, not a re-implementation. The ASGI app is
not booted; outbound HTTP is mocked and the DB is an isolated in-memory SQLite.
"""
import base64
import json
import time
import unittest.mock as mock
import pytest
# ── OAuth state signing ──────────────────────────────────────────
def test_oauth_state_round_trips_account_and_owner():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
payload = verify_oauth_state(state)
assert payload is not None
assert payload["a"] == "acct-123"
assert payload["o"] == "user@example.com"
assert payload["n"] # nonce present
def test_oauth_state_nonce_is_unique_per_call():
from routes.email_helpers import make_oauth_state, verify_oauth_state
a = verify_oauth_state(make_oauth_state("acct", "o"))
b = verify_oauth_state(make_oauth_state("acct", "o"))
assert a["n"] != b["n"]
def test_oauth_state_rejects_tampered_account_id():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload_str, sig = decoded.rsplit("|", 1)
payload = json.loads(payload_str)
payload["a"] = "evil-acct" # attacker swaps the target account
forged = base64.urlsafe_b64encode(
(json.dumps(payload, separators=(",", ":")) + "|" + sig).encode()
).decode()
assert verify_oauth_state(forged) is None
def test_oauth_state_rejects_forged_signature():
from routes.email_helpers import make_oauth_state, verify_oauth_state
state = make_oauth_state("acct-123", "user@example.com")
decoded = base64.urlsafe_b64decode(state.encode()).decode()
payload_str, _ = decoded.rsplit("|", 1)
forged = base64.urlsafe_b64encode((payload_str + "|" + "deadbeef" * 8).encode()).decode()
assert verify_oauth_state(forged) is None
@pytest.mark.parametrize("garbage", ["", "not-base64-at-all", "###", "a|b|c"])
def test_oauth_state_rejects_garbage(garbage):
from routes.email_helpers import verify_oauth_state
assert verify_oauth_state(garbage) is None
# ── _smtp_ready: OAuth accounts have no password but can still send ──
def test_smtp_ready_true_for_oauth_account_without_password():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.gmail.com",
"smtp_user": "me@nyu.edu",
"smtp_password": "",
"oauth_provider": "google",
}
assert _smtp_ready(cfg) is True
def test_smtp_ready_true_for_password_account():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.example.com",
"smtp_user": "me@example.com",
"smtp_password": "app-password",
"oauth_provider": "",
}
assert _smtp_ready(cfg) is True
def test_smtp_ready_false_without_password_or_oauth():
from routes.email_routes import _smtp_ready
cfg = {
"smtp_host": "smtp.example.com",
"smtp_user": "me@example.com",
"smtp_password": "",
"oauth_provider": "",
}
assert _smtp_ready(cfg) is False
def test_smtp_ready_false_without_host():
from routes.email_routes import _smtp_ready
cfg = {"smtp_host": "", "smtp_user": "me@x.com", "oauth_provider": "google"}
assert _smtp_ready(cfg) is False
# ── XOAUTH2 SASL framing ─────────────────────────────────────────
def test_xoauth2_raw_is_unencoded_sasl_frame():
from routes.email_helpers import _xoauth2_raw
assert _xoauth2_raw("me@nyu.edu", "tok123") == "user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
def test_xoauth2_bytes_is_raw_frame_encoded():
from routes.email_helpers import _xoauth2_bytes
assert _xoauth2_bytes("me@nyu.edu", "tok123") == b"user=me@nyu.edu\x01auth=Bearer tok123\x01\x01"
# ── Helpers for in-memory DB fixtures ────────────────────────────
def _make_db():
"""Return (Session, SessionFactory) backed by an isolated in-memory SQLite DB.
Used to test DB-touching helpers without the real database.
The factory lets tests open a fresh session after the helper closes its own.
"""
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from core.database import Base
engine = create_engine("sqlite:///:memory:", connect_args={"check_same_thread": False})
Base.metadata.create_all(engine)
Factory = sessionmaker(bind=engine)
return Factory(), Factory
def _make_account(session, account_id="acct-1", owner="alice", **kwargs):
"""Insert a minimal EmailAccount row and return it."""
from core.database import EmailAccount
row = EmailAccount(
id=account_id,
owner=owner,
name=kwargs.get("name", "Test"),
from_address=kwargs.get("from_address", "test@example.com"),
imap_host=kwargs.get("imap_host", "imap.gmail.com"),
imap_port=kwargs.get("imap_port", 993),
imap_user=kwargs.get("imap_user", "test@example.com"),
smtp_host=kwargs.get("smtp_host", "smtp.gmail.com"),
smtp_port=kwargs.get("smtp_port", 587),
smtp_user=kwargs.get("smtp_user", "test@example.com"),
)
for k, v in kwargs.items():
if hasattr(row, k):
setattr(row, k, v)
session.add(row)
session.commit()
return row
# ── Token encryption at rest ─────────────────────────────────────
def test_refresh_token_stored_encrypted_not_raw():
"""_refresh_google_token must encrypt the new access token before writing it
to the DB — storing the raw token string would expose credentials at rest."""
from src.secret_storage import encrypt as _enc, decrypt as _dec
from core.database import EmailAccount
raw_token = "ya29.test_access_token_raw"
db, Factory = _make_db()
_make_account(db, account_id="acct-r", owner="bob",
oauth_refresh_token=_enc("refresh-tok-xyz"))
db.close()
fake_resp = mock.MagicMock()
fake_resp.raise_for_status = mock.MagicMock()
fake_resp.json.return_value = {"access_token": raw_token, "expires_in": 3600}
with mock.patch("httpx.post", return_value=fake_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-r")
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-r").first()
stored = row.oauth_access_token
verify_db.close()
assert result == raw_token, "function should return the plain access token to callers"
assert stored != raw_token, "raw token must not be stored directly in the DB"
assert _dec(stored) == raw_token, "stored value must decrypt back to the raw token"
def test_refresh_stores_encrypted_expiry_not_token():
"""oauth_token_expiry stores only a timestamp, never the token value."""
from src.secret_storage import encrypt as _enc
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-e", owner="bob",
oauth_refresh_token=_enc("ref-tok"))
db.close()
fake_resp = mock.MagicMock()
fake_resp.raise_for_status = mock.MagicMock()
fake_resp.json.return_value = {"access_token": "ya29.secret", "expires_in": 3600}
with mock.patch("httpx.post", return_value=fake_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
_refresh_google_token("acct-e")
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-e").first()
expiry = row.oauth_token_expiry
verify_db.close()
assert "ya29" not in (expiry or ""), \
"token_expiry must be a timestamp, not the token string"
# ── Real OAuth callback route ─────────────────────────────────────
#
# These pull the actual google_oauth_callback endpoint out of the router and
# invoke it — they pin the real route's behaviour, not a re-implementation, so
# they fail if the ownership/state guards are ever removed or weakened.
def _callback_endpoint():
"""Return the live google_oauth_callback endpoint from the email router."""
from routes.email_routes import setup_email_routes
router = setup_email_routes()
for route in router.routes:
if route.path == "/api/email/oauth/google/callback" and "GET" in getattr(route, "methods", set()):
return route.endpoint
raise AssertionError("google_oauth_callback route not found")
class _FakeRequest:
"""Minimal stand-in for starlette Request — the callback only reads headers."""
headers = {"host": "localhost:7000"}
def _location(resp):
"""Pull the redirect target out of a RedirectResponse."""
return resp.headers["location"]
@pytest.mark.asyncio
async def test_callback_missing_code_returns_generic_error():
"""No `code` query param → generic error redirect, with no account id, owner,
or state echoed back into the URL."""
from routes.email_helpers import make_oauth_state
callback = _callback_endpoint()
state = make_oauth_state("acct-1", "alice")
resp = await callback(code=None, state=state, error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=missing_code" in loc
assert "acct-1" not in loc, "account id must not appear in redirect URL"
assert "alice" not in loc, "owner must not appear in redirect URL"
@pytest.mark.asyncio
async def test_callback_provider_error_returns_generic_error():
"""An `error` from Google → generic error redirect, no raw provider text."""
callback = _callback_endpoint()
resp = await callback(code=None, state=None, error="access_denied", request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=google_error" in loc
assert "access_denied" not in loc, "raw provider error must not leak into redirect"
@pytest.mark.asyncio
async def test_callback_tampered_state_returns_generic_error_no_leak():
"""Tampered/invalid state → invalid_state redirect; the auth code and any
token must never appear in the redirect URL."""
callback = _callback_endpoint()
resp = await callback(code="4/secret-auth-code", state="not-a-valid-state",
error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=invalid_state" in loc
assert "4/secret-auth-code" not in loc, "auth code must not leak into redirect"
assert "token" not in loc
@pytest.mark.asyncio
async def test_callback_owner_mismatch_does_not_write_tokens():
"""A signed, valid state whose owner does not match the target account's
owner must NOT write tokens — this blocks one authenticated user from
binding their Google account onto another user's mailbox row.
"""
from routes.email_helpers import make_oauth_state
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-x", owner="alice")
db.close()
# Token-exchange + userinfo would succeed — the point is the ownership gate
# rejects the write *before* trusting them.
token_resp = mock.MagicMock()
token_resp.raise_for_status = mock.MagicMock()
token_resp.json.return_value = {"access_token": "ya29.attacker", "refresh_token": "r", "expires_in": 3600}
userinfo_resp = mock.MagicMock()
userinfo_resp.is_success = True
userinfo_resp.json.return_value = {"email": "bob@evil.com", "name": "Bob"}
# State is genuinely signed, but for owner "bob" — not the row owner "alice".
state = make_oauth_state("acct-x", "bob")
with mock.patch("httpx.post", return_value=token_resp), \
mock.patch("httpx.get", return_value=userinfo_resp), \
mock.patch("core.database.SessionLocal", Factory):
callback = _callback_endpoint()
resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
loc = _location(resp)
assert "email_oauth_error=ownership_error" in loc
verify_db = Factory()
row = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-x").first()
token_after = row.oauth_access_token
verify_db.close()
assert token_after is None, "no token may be written when ownership check fails"
@pytest.mark.asyncio
async def test_callback_valid_owner_writes_encrypted_tokens_to_intended_account():
"""A signed state whose owner matches the target account writes the tokens —
and only to that account, stored encrypted (raw token never persisted)."""
from routes.email_helpers import make_oauth_state
from src.secret_storage import decrypt as _dec
from core.database import EmailAccount
db, Factory = _make_db()
_make_account(db, account_id="acct-v", owner="alice", imap_host="", smtp_host="")
_make_account(db, account_id="acct-other", owner="alice") # must stay untouched
db.close()
raw_access = "ya29.legit_access_token"
raw_refresh = "1//legit_refresh_token"
token_resp = mock.MagicMock()
token_resp.raise_for_status = mock.MagicMock()
token_resp.json.return_value = {"access_token": raw_access, "refresh_token": raw_refresh, "expires_in": 3600}
userinfo_resp = mock.MagicMock()
userinfo_resp.is_success = True
userinfo_resp.json.return_value = {"email": "alice@nyu.edu", "name": "Alice"}
state = make_oauth_state("acct-v", "alice")
with mock.patch("httpx.post", return_value=token_resp), \
mock.patch("httpx.get", return_value=userinfo_resp), \
mock.patch("core.database.SessionLocal", Factory):
callback = _callback_endpoint()
resp = await callback(code="4/code", state=state, error=None, request=_FakeRequest())
assert "email_oauth_success=1" in _location(resp)
verify_db = Factory()
target = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-v").first()
other = verify_db.query(EmailAccount).filter(EmailAccount.id == "acct-other").first()
verify_db.close()
assert target.oauth_provider == "google"
assert target.oauth_access_token != raw_access, "access token must be stored encrypted"
assert _dec(target.oauth_access_token) == raw_access
assert _dec(target.oauth_refresh_token) == raw_refresh
assert other.oauth_access_token is None, "tokens must only touch the intended account"
# ── Token refresh scenarios ───────────────────────────────────────
def test_get_valid_google_token_uses_cached_when_fresh():
"""_get_valid_google_token must NOT call refresh when the stored token is
still valid (expiry - 60s buffer > now). Refresh is an outbound HTTP call
that should only happen when genuinely needed."""
from src.secret_storage import encrypt as _enc
from routes.email_helpers import _get_valid_google_token
future_expiry = str(int(time.time()) + 7200) # 2 hours from now
cfg = {
"account_id": "acct-fresh",
"oauth_access_token": _enc("ya29.fresh_token"),
"oauth_token_expiry": future_expiry,
}
with mock.patch("routes.email_helpers._refresh_google_token") as mock_refresh:
result = _get_valid_google_token("acct-fresh", cfg)
assert result == "ya29.fresh_token"
mock_refresh.assert_not_called()
def test_get_valid_google_token_refreshes_when_expired():
"""_get_valid_google_token must call refresh when the token is expired."""
from src.secret_storage import encrypt as _enc
from routes.email_helpers import _get_valid_google_token
past_expiry = str(int(time.time()) - 10) # already expired
cfg = {
"account_id": "acct-exp",
"oauth_access_token": _enc("ya29.old_token"),
"oauth_token_expiry": past_expiry,
}
with mock.patch("routes.email_helpers._refresh_google_token", return_value="ya29.new_token") as mock_refresh:
result = _get_valid_google_token("acct-exp", cfg)
mock_refresh.assert_called_once_with("acct-exp")
assert result == "ya29.new_token"
def test_refresh_failure_returns_none_no_secret_raised():
"""When the refresh HTTP call fails, _refresh_google_token must return None
silently. It must not raise an exception or surface token/secret details."""
from src.secret_storage import encrypt as _enc
db, Factory = _make_db()
_make_account(db, account_id="acct-fail", owner="dave",
oauth_refresh_token=_enc("ref-tok"))
db.close()
failing_resp = mock.MagicMock()
failing_resp.raise_for_status.side_effect = Exception("401 Unauthorized")
with mock.patch("httpx.post", return_value=failing_resp), \
mock.patch("core.database.SessionLocal", Factory), \
mock.patch("routes.email_helpers.os.environ.get", side_effect=lambda k, d="": {
"GOOGLE_OAUTH_CLIENT_ID": "cid", "GOOGLE_OAUTH_CLIENT_SECRET": "csec"
}.get(k, d)):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-fail")
assert result is None, "failed refresh must return None, not raise"
def test_refresh_without_credentials_returns_none():
"""_refresh_google_token must return None immediately when the OAuth client
credentials are not configured — no DB query, no HTTP call."""
with mock.patch("routes.email_helpers.os.environ.get", return_value=""):
from routes.email_helpers import _refresh_google_token
result = _refresh_google_token("acct-any")
assert result is None
# ── Password-account regression ───────────────────────────────────
def test_imap_connect_uses_login_for_password_accounts():
"""Existing password-auth IMAP accounts must still call conn.login() and
must NOT trigger the XOAUTH2 authenticate path."""
from routes.email_helpers import _imap_connect
mock_conn = mock.MagicMock()
# _imap_connect calls _get_email_config internally — mock it to return our cfg.
cfg = {
"imap_host": "imap.gmail.com",
"imap_port": 993,
"imap_starttls": False,
"imap_user": "me@gmail.com",
"imap_password": "app-password-xyz",
"oauth_provider": "",
"account_id": "acct-pw",
}
with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
_imap_connect("acct-pw", owner="alice")
mock_conn.login.assert_called_once_with("me@gmail.com", "app-password-xyz")
mock_conn.authenticate.assert_not_called()
def test_imap_connect_uses_xoauth2_for_oauth_accounts():
"""OAuth accounts must call conn.authenticate('XOAUTH2', ...) and must NOT
call conn.login() — which would fail without a password."""
from routes.email_helpers import _imap_connect
from src.secret_storage import encrypt as _enc
mock_conn = mock.MagicMock()
future_expiry = str(int(time.time()) + 7200)
cfg = {
"imap_host": "imap.gmail.com",
"imap_port": 993,
"imap_starttls": False,
"imap_user": "me@nyu.edu",
"imap_password": "",
"oauth_provider": "google",
"account_id": "acct-oauth",
"oauth_access_token": _enc("ya29.live_token"),
"oauth_token_expiry": future_expiry,
}
with mock.patch("routes.email_helpers._open_imap_connection", return_value=mock_conn), \
mock.patch("routes.email_helpers._get_email_config", return_value=cfg):
_imap_connect("acct-oauth", owner="alice")
mock_conn.authenticate.assert_called_once()
assert mock_conn.authenticate.call_args[0][0] == "XOAUTH2"
mock_conn.login.assert_not_called()
@pytest.mark.asyncio
async def test_account_list_response_does_not_expose_token_values():
"""The /accounts list route is the client-facing account inventory. It must
expose `oauth_provider` (so the UI can show OAuth status) but never the
access/refresh token values, encrypted or otherwise — only boolean
has_*_password flags and the provider name."""
from routes.email_routes import setup_email_routes
from src.secret_storage import encrypt as _enc
raw_access = "ya29.super_secret_access_token"
raw_refresh = "1//super_secret_refresh_token"
db, Factory = _make_db()
_make_account(db, account_id="acct-list", owner="alice",
oauth_provider="google",
oauth_access_token=_enc(raw_access),
oauth_refresh_token=_enc(raw_refresh))
db.close()
router = setup_email_routes()
list_accounts = None
for route in router.routes:
if route.path == "/api/email/accounts" and "GET" in getattr(route, "methods", set()):
list_accounts = route.endpoint
break
assert list_accounts is not None, "accounts list route not found"
with mock.patch("core.database.SessionLocal", Factory):
result = await list_accounts(owner="alice")
blob = json.dumps(result)
assert raw_access not in blob, "raw access token must not appear in list response"
assert raw_refresh not in blob, "raw refresh token must not appear in list response"
assert _enc(raw_access) not in blob, "encrypted token must not be sent to the client either"
acct = result["accounts"][0]
assert acct["oauth_provider"] == "google" # status is exposed
assert "oauth_access_token" not in acct # token value is not
assert "oauth_refresh_token" not in acct