mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-06-15 17:25:26 -04:00
nsgds
7ae6133d7f
* fix(agent): don't let a materialized default budget defeat context scaling #1230 scales agent_input_token_budget to the model's context window unless the user explicitly set a budget, detected via is_setting_overridden(). But the settings-save path materializes every DEFAULT_SETTINGS key into settings.json (load_settings merges defaults; handlers persist the merged dict), so the persisted default 6000 reads as "overridden" and the budget code takes the min(6000, ctx) branch — silently re-capping long-context models at 6000 for anyone who has ever saved a setting. This reintroduces the exact regression #1170/#1230 set out to fix. Add is_setting_customized() (saved value != default) and gate the scaling on it instead of mere presence. A persisted default is not a user choice. is_setting_overridden has exactly one consumer (this budget path), so the change is contained. Tests cover the materialized-default regression, a deliberately-chosen budget still being honoured, and the absent-key case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): rework context-budget fix per review (#4122) Address RaresKeY's review: P2 (explicitness): is_setting_customized treated a saved value equal to the default as "not explicit", which ALSO blocked a user from deliberately pinning the default budget. Reframe the default value itself as the AUTO sentinel — agent_input_token_budget == DEFAULT_BUDGET means "scale to the model's context window", any other value is an explicit cap. A materialized default still reads as auto (fixing the original regression), and any non-default value the user chooses is now honoured. Drop the now-unused is_setting_customized helper. P2 (fallback context): auto-scaling trusted get_context_length() even when it returned only the bare DEFAULT_CONTEXT fallback (no endpoint-reported / known window), over-allocating on self-hosted/proxy setups. Add get_context_length_known() (also returns whether the window was actually discovered); the budget block passes 0 when unknown so auto-scaling stays conservative instead of inflating to an unproven window. hard_max stays auto-only — a deliberate explicit budget wins (#1190); kept that contract and answered the reviewer's question rather than silently reversing it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(agent): lock the materialized-default budget regression (review on #4121) Per WGlynn's review on the issue: add an end-to-end regression that saves an UNRELATED setting (which makes the settings-save path materialize the budget default into settings.json) and asserts the budget still auto-scales rather than re-reading as an explicit 6000 cap — locking the exact reopening shut. To make the test bite the production decision (not just re-derive it), extract `budget_is_explicit()` into src/context_budget.py and use it from the agent loop. It keys off value-vs-default (the default is the auto sentinel), NOT settings presence — which is the whole point, since the save path materializes defaults. Note: after this PR's rework, is_setting_overridden has ZERO production callers, so the merged-dict materialization smell can't reach any setting through a presence check today (WGlynn's durability concern). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agent): bind the budget context window to its own provenance (review #4122) RaresKeY caught a correctness bug in the fallback-context guard: stream_agent_loop kept only the `known` flag from get_context_length_known() and budgeted off the passed-in `context_length`, which can come from a *different* lookup. Two failures: - local endpoints are re-queried, so the passed value can be a stale DEFAULT_CONTEXT fallback while the fresh probe proves the real (smaller) served context — we'd scale off the stale value; - callers that don't pass context_length (scheduled tasks, teacher escalation, skill test runs, bg_monitor) were capped at 6000 even when a long window is discoverable. Extract budget_context_for_model() which returns the freshly-probed window when known else 0, binding the flag to the value it proves; the agent loop uses it. Regression tests cover the stale-fallback, no-arg-caller, and probe-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): fix stale budget comments + tighten to the contract (review #4122) - settings.py: an explicit budget is clamped to the window only — hard_max is auto-only (#1190); drop the incorrect "and to hard_max". - is_setting_overridden docstring: drop the stale "adaptive budgets" example; point value-sensitive callers at context_budget.budget_is_explicit. - Tighten the budget-block comments to the contract (default = auto sentinel, non-default = explicit cap, hard_max = auto-only ceiling). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agent): correct budget issue citations (#1190 → merged #1230/#1273) The context-budget contract (auto-sentinel, explicit budgets honoured, hard_max auto-only) merged via #1230 — #1190 was the earlier, closed, superseded PR. Re-point the contract comments at #1230 (the live source, already cited for the auto-sentinel two lines up in settings.py). The configurable hard_max setting (`agent_input_token_hard_max`) was a reviewer requirement first raised on #1190, omitted from the merged #1230, and actually added in #1273 — credit #1273 for it and correct the test comment's history (it previously implied this PR completed the requirement). Comment/docstring-only; no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
59 lines
2.0 KiB
Python
59 lines
2.0 KiB
Python
"""Regression coverage for llama-server style /v1 model-list endpoints (#3330)."""
|
|
|
|
import httpx
|
|
|
|
from src import endpoint_resolver, llm_core, model_context
|
|
|
|
|
|
def test_build_models_url_accepts_v1_base_and_chat_url(monkeypatch):
|
|
monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
|
|
|
|
assert (
|
|
endpoint_resolver.build_models_url("http://127.0.0.1:8080/v1")
|
|
== "http://127.0.0.1:8080/v1/models"
|
|
)
|
|
assert (
|
|
endpoint_resolver.build_models_url("http://127.0.0.1:8080/v1/chat/completions")
|
|
== "http://127.0.0.1:8080/v1/models"
|
|
)
|
|
|
|
|
|
def test_llm_core_list_model_ids_queries_models_for_v1_base(monkeypatch):
|
|
monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
|
|
monkeypatch.setattr(llm_core, "_configured_cached_model_ids", lambda url, **kwargs: [])
|
|
seen = []
|
|
|
|
def fake_get(url, headers=None, timeout=None):
|
|
seen.append(url)
|
|
request = httpx.Request("GET", url)
|
|
return httpx.Response(200, json={"data": [{"id": "qwen3"}]}, request=request)
|
|
|
|
monkeypatch.setattr(llm_core.httpx, "get", fake_get)
|
|
|
|
assert llm_core.list_model_ids("http://127.0.0.1:8080/v1", timeout=1) == ["qwen3"]
|
|
assert seen == ["http://127.0.0.1:8080/v1/models"]
|
|
|
|
|
|
def test_model_context_queries_models_for_v1_base(monkeypatch):
|
|
monkeypatch.setattr(endpoint_resolver, "resolve_url", lambda url: url)
|
|
seen = []
|
|
|
|
def fake_get(url, timeout=None):
|
|
seen.append(url)
|
|
request = httpx.Request("GET", url)
|
|
if url.endswith("/slots"):
|
|
return httpx.Response(404, request=request)
|
|
return httpx.Response(
|
|
200,
|
|
json={"data": [{"id": "qwen3", "context_length": 32768}]},
|
|
request=request,
|
|
)
|
|
|
|
monkeypatch.setattr(model_context.httpx, "get", fake_get)
|
|
|
|
assert model_context._query_context_length("http://127.0.0.1:8080/v1", "qwen3") == (32768, True)
|
|
assert seen == [
|
|
"http://127.0.0.1:8080/slots",
|
|
"http://127.0.0.1:8080/v1/models",
|
|
]
|