fix(model-routes): harden _probe_endpoint against malformed model-list responses (#4789)

* fix(model-routes): harden _probe_endpoint against malformed model-list responses

_probe_endpoint parsed model lists with data.get(...) at four sites without
checking that data is a dict, and built the list with a truthiness-only
filter. A /models (or /api/tags) endpoint returning HTTP 200 with valid but
non-dict JSON ([], "x", null, 123) made data.get(...) raise AttributeError,
and a non-string id like 123 passed the filter and then hit .startswith() /
.lower() in the Z.AI/Kimi curated merge and _is_chat_model(). Both errors are
swallowed by the broad except Exception, but the comprehension dies mid-list
so the ENTIRE probed model list is discarded and the endpoint silently
degrades — masking a misconfigured/non-compliant upstream as "no models".

- Guard each data.get(...) with isinstance(data, dict) so a non-dict body
  falls through the existing `or []` default.
- Restrict the OpenAI and Ollama model-list comprehensions to non-empty str
  values, protecting the .startswith() merges and both _is_chat_model calls.
- Add an isinstance guard at the top of _is_chat_model (defense in depth for
  all four call sites).

No behavior change for well-formed {"data":[...]} / {"models":[...]}
responses. Adds regression tests (non-dict body via caplog, mixed/all
non-string ids, _is_chat_model boundary) that fail before the fix and pass
after.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(model-routes): extract _openai_model_ids / _ollama_model_names helpers

Per review on #4789: the malformed-response guards were inlined four times in
_probe_endpoint (two OpenAI-id comprehensions, two Ollama-name comprehensions).
Pull each into a small, directly-testable helper so the security-relevant
parsing lives in one place and a future malformed-shape fix doesn't have to be
applied in four spots (CONTRIBUTING flags repeated logic for this reason).

Behavior is unchanged. Adds direct unit tests for both helpers (non-dict body,
non-string ids, non-dict entries, name>model precedence).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Solanki Sumit
2026-06-24 22:35:31 +05:30
committed by GitHub
parent 4e46e415ea
commit 22379fe736
3 changed files with 108 additions and 4 deletions
+36 -4
View File
@@ -523,6 +523,10 @@ _NON_CHAT_EXACT_PREFIXES = (
def _is_chat_model(model_id: str) -> bool:
"""Return True if the model ID looks like a chat/completions-capable model."""
if not isinstance(model_id, str):
# Non-compliant upstreams can return non-string IDs (e.g. int/None);
# treat them as chat-capable rather than crashing on .lower().
return True
mid = model_id.lower()
for prefix in _NON_CHAT_PREFIXES:
if mid.startswith(prefix):
@@ -726,6 +730,34 @@ def _is_loading_model_response(resp: Any) -> bool:
def _openai_model_ids(data: Any) -> List[str]:
"""Extract OpenAI-style model IDs (``{"data": [{"id": ...}]}``).
Tolerates a non-dict body and non-string IDs from non-compliant upstreams,
returning only non-empty string IDs.
"""
items = data.get("data") if isinstance(data, dict) else None
return [m["id"] for m in (items or [])
if isinstance(m, dict) and isinstance(m.get("id"), str) and m["id"]]
def _ollama_model_names(data: Any) -> List[str]:
"""Extract native-Ollama model names (``{"models": [{"name"|"model": ...}]}``).
Same tolerance as :func:`_openai_model_ids`: a non-dict body or non-string
value is skipped rather than crashing, preserving name-then-model precedence.
"""
items = data.get("models") if isinstance(data, dict) else None
out: List[str] = []
for m in (items or []):
if not isinstance(m, dict):
continue
v = m.get("name") or m.get("model")
if isinstance(v, str) and v:
out.append(v)
return out
def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> List[str]:
"""Probe a base URL's /models endpoint and return list of model IDs.
For Anthropic, queries their /v1/models API, falling back to hardcoded list."""
@@ -748,7 +780,7 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
r = httpx.get(url, headers=headers, timeout=timeout, verify=llm_verify())
r.raise_for_status()
data = r.json()
models = [m.get("id") for m in (data.get("data") or []) if m.get("id")]
models = _openai_model_ids(data)
if models:
return models
except httpx.HTTPStatusError as e:
@@ -770,10 +802,10 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
r.raise_for_status()
data = r.json()
# OpenAI format: {"data": [{"id": "model-name"}]}
models = [m.get("id") for m in (data.get("data") or []) if m.get("id")]
models = _openai_model_ids(data)
# Ollama format: {"models": [{"name": "model-name"}]}
if not models:
models = [m.get("name") or m.get("model") for m in (data.get("models") or []) if m.get("name") or m.get("model")]
models = _ollama_model_names(data)
if models:
# Z.AI coding plan omits some working models from /models;
# append curated-only entries for that endpoint only.
@@ -812,7 +844,7 @@ def _probe_endpoint(base_url: str, api_key: str = None, timeout: int = 5) -> Lis
r = httpx.get(root + "/api/tags", timeout=timeout, verify=llm_verify())
r.raise_for_status()
data = r.json()
models = [m.get("name") or m.get("model") for m in (data.get("models") or []) if m.get("name") or m.get("model")]
models = _ollama_model_names(data)
if models:
return [m for m in models if _is_chat_model(m)]
except Exception as e: