Commit Graph

125 Commits

Author SHA1 Message Date
Husam f23221420f fix(skills): replace deprecated utcnow in skill timestamp helper (#5777)
* fix(skills): replace deprecated utcnow in skill timestamp helper

_now_iso() builds the 'created' value in skill frontmatter. datetime.utcnow()
returns a naive datetime and has been deprecated since Python 3.12, scheduled
for removal. Switch to the timezone-aware datetime.now(timezone.utc), keeping
the serialized YYYY-MM-DDTHH:MM:SSZ shape unchanged so existing skill files
keep parsing.

timezone.utc is used rather than the datetime.UTC alias, which is 3.11+ only.

Adds regression tests covering the deprecation, the serialized shape, and
UTC correctness under a non-UTC local timezone -- the last guards against a
bare datetime.now(), which yields the same shape but local wall time.

Fixes #5697

* test(skills): skip timezone mutation where unsupported

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-30 09:54:59 +01:00
Boody 2e631ad816 improve cache size calculation by filtering file types 2026-07-29 12:47:55 +03:00
Boody 9914651cc9 improve cache eviction logic to handle file access errors and ensure stability 2026-07-29 12:41:31 +03:00
Boody 5104a9a967 feat(tts): implement TTS cache size limit and eviction policy 2026-07-28 21:34:03 +03:00
pewdiepie-archdaemon d8a2059df8 Merge verified Odysseus fixes 2026-07-23 14:49:02 +00:00
Afonso Coutinho 1aad1db9f6 fix: services research source extraction crashes on a non-dict finding (#1868) 2026-07-20 09:39:16 +02:00
Emir Çoban 45b6771e73 fix(skills): block private SSRF targets and revalidate redirects in importer (#5261)
* Harden skill importer against SSRF: block private targets + revalidate redirects per hop

The skill importer validated only the initial URL with the lenient SSRF guard
(block_private=False) and then fetched with follow_redirects=True, so a 3xx to
an internal/metadata address (169.254.169.254, 127.0.0.1, RFC-1918) was still
connected to — inconsistent with the hardened services/search/content.py
:_get_public_url path.

Add a _get_checked() helper that follows redirects manually and re-runs the
SSRF guard with block_private=True on every hop, and route all three fetch
sites (skills.sh unwrap, _fetch_bytes, _list_github_dir) through it. GitHub's
own redirects and the final-host _assert_github_url checks are preserved.

Adds hermetic regression tests (IP-literal hosts, faked HTTP layer) and updates
the existing mock signature for the new block_private kwarg.

Defense-in-depth: the endpoint is admin-gated (require_admin) and admins are
trusted per THREAT_MODEL.md, so this is not a cross-boundary vulnerability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: enforce follow_redirects=False invariant in mock client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 08:33:10 -06:00
Afonso Coutinho a4e66bb59f fix: TTS available crashes on non-string tts_provider (#2034) 2026-07-11 03:19:51 +01:00
Afonso Coutinho 06e038f00e fix: hwfit params_b/is_prequantized crash on non-string catalog fields (#2094) 2026-07-11 03:00:28 +01:00
pewdiepie-archdaemon 4c24d5d9a6 Merge remote-tracking branch 'origin/dev'
# Conflicts:
#	routes/contacts_routes.py
2026-07-07 00:51:34 +00:00
pewdiepie-archdaemon a1a14bd5c9 Checkpoint Odysseus local update 2026-07-07 00:50:07 +00:00
Ohualtex 15239e34c3 fix(search): extract non-ASCII capitalized names in _extract_entities
_extract_entities used the ASCII-only class [A-Z][a-zA-Z]+ to pull name
entities from a query, so non-ASCII names were dropped ("İstanbul",
"Zürich" yielded nothing) or shredded ("São Paulo" -> only "Paulo"),
degrading query enhancement for non-English/accented searches. Match
Unicode words and keep the alphabetic, uppercase-initial ones; ASCII
behaviour (the word boundary already excludes camelCase mid-word
capitals) is unchanged.
2026-07-04 20:42:06 +03:00
Ernest Hysa c76d5e6ab4 fix(search): pin httpx connection to resolved IP to block DNS rebinding (#704)
* fix(search): pin DNS-validated fetch connections

Rebase the DNS-rebinding SSRF fix onto current dev after search content moved behind the services.search.content canonical module.

Integrate the pinned httpcore NetworkBackend/BaseTransport approach with the current size-capped Client.stream fetch path, preserving Host/SNI semantics while forcing TCP connect to the already validated resolved IP.

Keep src.search.content as the compatibility wrapper and preserve existing OG-image http(s) behavior; this avoids reintroducing the unrelated scope changes that previously blocked review.

Add the explicit httpcore>=1.0,<2.0 requirement used by the public httpcore NetworkBackend and ConnectionPool APIs.

* test(search): restore and rebase DNS rebinding regressions

Keep the current security regression coverage that the stale PR branch had deleted, including auth-disabled localhost bypass and Ollama cookbook hardening tests.

Carry forward the DNS-rebinding coverage for private resolve blocking, pinned TCP connect behavior, Host header preservation, redirect revalidation, and the BaseTransport/public-httpcore static guard.

Update redirect tests to mock the current Client.stream-based capped fetch path rather than the older httpx.stream/get path.

* test(search): adapt size-cap fetch tests to pinned client stream

The DNS-rebinding repair moved _get_public_url from the module-level httpx.stream shortcut to httpx.Client(...).stream(...) so the fetch can use the pinned transport.

Keep the existing size-cap test fakes by routing Client.stream through the monkeypatched httpx.stream only when a test has installed that fake; otherwise fall back to a real Client.

This fixes the CI failures in tests/test_web_fetch_size_caps.py without touching unrelated upload-handler atomicity behavior, which is already flaky on clean origin/dev.

---------

Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-02 13:08:06 +01:00
pewdiepie-archdaemon 0b4ef7187f Stabilize chat and cookbook workflows 2026-07-01 10:09:25 +00:00
red person ddc989129c Ignore invalid serve profile inputs (#1827) 2026-06-29 18:47:19 +01:00
Joel Alejandro Escareño Fernández 70d5ca86be feat(catalog): add Gemma 4 12B/QAT entries and RTX 3050 bandwidth (#4728)
Add official Gemma 4 12B-it plus QAT-INT4/INT8 catalog entries (with their
GGUF sources), QAT quantization support across the quant tables and the
prequantized-prefix list, and the missing RTX 3050 / 3050 Ti memory
bandwidth so speed estimates stop falling back to the generic cuda value.
2026-06-23 18:23:46 +02:00
MACKAT05 08e1d05f35 fix(hwfit): repair remote Windows hardware scan over SSH (#4674)
Remote Cookbook hwfit probes failed on Windows hosts because the PowerShell script was sent as nested -Command quoting through OpenSSH. Use -EncodedCommand for remote probes, auto-detect platform when omitted (including Darwin for Mac SSH hosts), and return a clearer error when SSH works but the probe fails.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-22 20:59:09 +02:00
pewdiepie-archdaemon 324b1d9eaf Merge origin/dev into main 2026-06-21 11:08:50 +00:00
pewdiepie-archdaemon 2be9d8efd4 Cookbook backend detection: report Vulkan on AMD hosts without ROCm; gate CUDA build on actual NVIDIA hardware
Three classes of incorrect detection fixed:

(1) AMD GPU + no ROCm installed (e.g. Strix Halo) was reported as
    backend=rocm everywhere, so launch commands emitted
    HIP_VISIBLE_DEVICES (silent no-op on Vulkan) and the from-source
    build path failed. Both _probe_amd_sysfs (routes/cookbook_routes)
    and _detect_amd (services/hwfit/hardware) now probe rocminfo /
    hipconfig / vulkaninfo at detection time and report vulkan when
    only Vulkan is present.

(2) Build helper was picking the CUDA branch on AMD hosts whenever a
    stray pip-installed nvcc was on PATH (vLLM wheels carry one
    without libcudart). Added _odysseus_has_nvidia_hw() that checks
    nvidia-smi / /dev/nvidia* / lspci, and gates both the nvcc PATH
    augmentation and the CUDA elif branch on real hardware.

(3) Build chain reordered to ROCm/HIP > CUDA > Vulkan > CPU. Vulkan
    tier added between CUDA and CPU as a portable fallback for hosts
    with a GPU but no native toolchain (the common Strix Halo case).
    Same _append_llama_cpp_linux_accel_build_lines also auto-attempts
    sudo -n apt/pacman/dnf install of cmake/build-essential/git when
    they are missing, surfacing a clear no-passwordless-sudo warning
    otherwise.
2026-06-19 00:33:07 +00:00
RaresKeY 67b40d65ad fix(hwfit): normalize CPU arch for fallback estimates (#4441) 2026-06-18 20:26:22 +02:00
Catalin Iliescu 4e0d6eb57b fix(hwfit): use CPU fallback for cpu_only speed estimates (#4397)
* fix(hwfit): use CPU fallback for cpu_only speed estimates

* fix(hwfit): preserve ARM fallback for cpu_only estimates

---------

Co-authored-by: Cata <cata@bigjohn.local>
2026-06-16 14:18:31 +00:00
Kenny Van de Maele 3409f7f108 refactor(search): centralize the web-scraping User-Agent into one constant (#4325)
The outbound UA for web_fetch / web_search was inlined in four places with
two different values and nothing keeping them current: content.py pinned a
mid-2021 Chrome 91 build, and providers.py sent a bare Mozilla/5.0 in three
spots. Some sites serve a degraded or blocked page to a UA that old.

Add WEB_FETCH_USER_AGENT to src/constants.py (env-overridable, matching the
existing Copilot/Kimi UA-constant pattern) and import it in content.py and
providers.py. Default to a current, common desktop UA so pages return their
normal HTML: the market-leading desktop OS (Windows; NT 10.0 covers Windows
10 and 11) and browser (Chrome) on a current stable build. The version is now
bumped in one place.

Service-specific self-identifying agents (Copilot, Kimi, webhooks, cookbook)
are intentionally left separate. Adds a regression pinning the constant shape,
the env override, and a guard against a new inline Mozilla literal in the
search sources.

Closes #4324
2026-06-16 01:33:47 +00:00
Kenny Van de Maele 5f301a45da fix(search): add download budgets to web_fetch with truncation notice and hard ceiling (#3955)
* fix(search): add download budgets to web_fetch with truncation notice and hard ceiling

MAX_OUTPUT_CHARS only trims what the agent sees; fetch_webpage_content
buffered and cached the entire response body first, so a large or hostile
URL could pull arbitrarily many bytes into memory and the content cache.

The fetch is now a capped streaming GET (SSRF redirect guard unchanged):
a soft default budget (WEB_FETCH_SOFT_MAX_BYTES, 2 MB), a per-call
override via full/max_bytes on the web_fetch tool, and a hard ceiling
(WEB_FETCH_HARD_MAX_BYTES, 20 MB) that the override can never exceed.
When Content-Length already declares a body over the ceiling the fetch
is refused before any body bytes are buffered. Truncated results carry
truncated/fetched_bytes/total_bytes, the tool output leads with a
partial-content notice telling the model how to re-fetch with full=true,
and the tool schema documents the flag. A truncated PDF is reported as
a budget error since a cut PDF is unparseable. The effective cap is part
of the content-cache key so a truncated fetch is never served to a
full-budget request.

Existing tests that faked httpx.get or the old _get_public_url signature
are adapted to the streaming interface; behavior pins are unchanged.

Fixes #3812

* fix(search): close compressed-body cap bypass and protect the partial notice

Addresses RaresKeY's review on #3955:

- Force Accept-Encoding: identity for the capped fetch. With gzip/deflate the
  wire bytes (and Content-Length) can be a fraction of the decoded body, so a
  tiny compressed response could pass the hard-cap preflight and then expand
  past the ceiling in a single decoded chunk before the streamed cap could
  slice it. Identity makes Content-Length the true body size and keeps each
  streamed chunk bounded by the network read, so the hard ceiling actually
  bounds memory.
- Lead web_fetch output with the partial-content notice and cap the page
  title. The notice is the user-facing contract for partial fetches, but the
  title is untrusted, uncapped page content; placed ahead of the notice a giant
  title could push it past MAX_OUTPUT_CHARS and drop it. The notice now leads
  and the title is capped as a second guard.

Adds regressions: the fetch advertises identity encoding, and a truncated
result with an oversized title still surfaces the partial notice.

* fix(search): reject compressed responses that ignore the identity request

Requesting Accept-Encoding: identity is not enough on its own: a server can
ignore it and still return Content-Encoding: gzip, and httpx.iter_bytes would
decode that, so a tiny compressed body could balloon into one decoded chunk
far past the hard cap before the streamed loop slices it (and Content-Length,
the compressed wire length, makes the preflight and size metadata unreliable).

Refuse a non-identity Content-Encoding before reading the body. Adds a
regression where the server ignores the identity request and returns gzip;
the fetch is refused before any body is decoded.
2026-06-15 17:38:09 +00:00
Kenny Van de Maele 760c8036ae refactor(search): import REQUEST_TIMEOUT from constants in providers.py (#4331)
providers.py redefined REQUEST_TIMEOUT = 20 locally, shadowing the same
value in src/constants.py and risking drift if the constant is bumped.
Import it from src.constants and drop the local copy; same value, one
source of truth.

Closes #4329
2026-06-15 17:22:08 +00:00
darius-f96 ed8b6a1688 fix(hwfit): add GB10 unified-memory bandwidth so speed scores are real (#4270)
NVIDIA Grace Blackwell GB10 / DGX Spark was missing from GPU_BANDWIDTH, so
_lookup_bandwidth() returned None for it and _estimate_speed() fell through
to the crude FALLBACK_K path (k/active-params). That over-stated tok/s and
let speed scores saturate regardless of the box's real ~273 GB/s LPDDR5X
pool — distorting model ranking on these 128GB unified-memory rigs.

Add "gb10": 273 (GB/s). nvidia-smi reports the device name as "NVIDIA GB10",
which substring-matches the new key, so detected GB10 boxes now estimate
speed from the real bandwidth instead of the fallback.
2026-06-15 18:55:15 +02:00
Kenny Van de Maele 7a5442d74d test(hwfit): fix non-Apple guard to assert the Apple matcher (unblocks pytest gate) (#4303)
* test(hwfit): assert the Apple matcher, not the general lookup, in the non-Apple guard

b80bc75 (#2564) added test_non_apple_gpu_with_cores_does_not_match, which
asserts _lookup_bandwidth(RTX 4090) is None. But '4090': 1008 has been in
the general GPU_BANDWIDTH table since v1.0, so _lookup_bandwidth correctly
returns the card's real bandwidth and the test fails (expected None, got
1008) - reddening the required pytest gate on dev and, by inheritance,
every open PR.

The guard's actual intent is that the Apple-specific bandwidth path does
not false-match a non-Apple card that carries a gpu_cores count. Point
the two asserts at _lookup_apple_bandwidth, which returns None for any
name without 'apple' regardless of the general table. The general-lookup
behavior (4090 -> 1008) is correct and untouched.

* fix(hwfit): route string GPU names through the Apple bandwidth helper

Second half of the #2564 regression (RaresKeY review on #4303). That
change moved the Apple tiers out of the generic GPU_BANDWIDTH table into
the dict-only _lookup_apple_bandwidth, but _lookup_bandwidth only called
that helper for dict inputs. A bare-string caller like
_lookup_bandwidth("Apple M3 Max") therefore fell through to the generic
table, found no Apple key, and returned None instead of the conservative
tier. Route both dict and string inputs through the Apple helper (a
string carries no gpu_cores, so it gets the model's lowest tier).
Regression added for the string path plus a non-Apple string control.
2026-06-15 14:01:05 +00:00
Ahmad Naalweh b80bc75984 fix(hwfit): distinguish Apple Silicon bandwidth variants (#2564)
* fix: resolve Apple Silicon bandwidth variants

* fix(hwfit): preserve string lookup path in _lookup_bandwidth

* fix(hwfit): guard Apple bandwidth lookup against false GPU matches

Add "apple" not in gn check to _lookup_apple_bandwidth() so that
non-Apple GPUs with "m3"/"m4"/"m5" in their names (e.g. NVIDIA
Quadro M4 000) don't incorrectly match Apple bandwidth tiers.

Addresses @o3LL review comment on PR #2564.
2026-06-15 15:13:03 +02:00
andrewemer 602ea4fc40 fix(agent): skill-prescribed tools never reach the model's schema list (#4008)
* Agent: make skill-prescribed tools actually callable

The skill index and matched-skill procedures are injected into the
prompt, but tool selection never followed: manage_skills wasn't in the
RAG-selected schema list (so the model substituted manage_memory), and
a matched skill could prescribe tools (grep, read_file) the model had
no schema for. Now:

- manage_skills rides along whenever the owner has any skills indexed
- a Jaccard-matched skill's requires_toolsets join the selection
- viewing a skill mid-turn via manage_skills unlocks its
  requires_toolsets for subsequent rounds
- admin-intent turns send _ADMIN_TOOLS schemas, matching the prompt
  text _build_base_prompt already advertises
- index_for(active_toolsets=None) no longer hides requires_toolsets
  skills from callers that don't know the active set

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Agent: validate skill requires_toolsets against known tools, not TOOL_SECTIONS

grep/glob/ls ship as function schemas without a prompt-prose section,
so gating on TOOL_SECTIONS silently dropped them from a skill's
requires_toolsets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 20:32:43 +09:00
Vishnu ac624d2711 fix(memory): reject ambiguous multi-object outputs during skill extraction (#3985) 2026-06-15 10:44:43 +00:00
Karthik Rajesh 7c976fa7cc feat(cookbook): surface Docker hardware visibility warnings (#3658) 2026-06-15 15:51:04 +09:00
holden093 1a77dbf445 fix(youtube): consolidate duplicate handler
Make src.youtube_handler a compatibility wrapper around services.youtube.youtube_handler so transcript state, URL parsing, and timeout behavior no longer diverge.
2026-06-15 15:03:41 +09:00
Ashvin 69f276774d fix(hwfit): serve profiles for sub-8192 context models
Allow serve-profile generation for models whose trained context window is below 8192 while preserving the 8K shrink floor for larger models.
2026-06-15 15:02:22 +09:00
Kenny Van de Maele d91fb9be51 fix(search): read plain-text, Markdown, and JSON URLs in fetch_webpage_content (#3809)
raw.githubusercontent.com serves Markdown as text/plain, JSON APIs and raw
config files serve application/json, and a lot of code and tool documentation
lives in .md/.txt. fetch_webpage_content only handled PDF and HTML, so a
non-HTML body produced empty content and web_fetch reported 'no readable text
content'. Add a branch that returns the body verbatim for non-HTML text/*,
JSON (application/json and +json), and a .md/.txt/.text/.json URL-suffix
fallback for mislabeled octet-stream. HTML and PDF handling unchanged.

Fixes #3808
2026-06-11 14:24:53 +00:00
ThomasAngel 9baa2ab561 chore: Switch duckduckgo-search to ddgs (#3143)
* Switch to ddgs

duckduckgo_search was deprecated, this is the recommended replacement

* Update test_service_search_provider_guards.py

According to review comment
2026-06-10 17:59:47 +02:00
ooovenenoso 46c5439461 fix(research): track analyzed URLs separately (#3125)
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
2026-06-10 12:08:22 +01:00
pewdiepie-archdaemon b5b509d9c0 Merge remote-tracking branch 'origin/main' into dev 2026-06-09 10:41:48 +09:00
Boody ee88736a22 fix: Enforce dynamic custom search result limits in backend (#2359)
* fixed confusing credentials prompt

* fix(setup): return status from create_default_admin function

* fix(setup): initialize admin creation status in main function

* fix(setup): enhance admin creation feedback and status handling

* Enhance admin user login messages with conditional feedback based on creation status

* Refine admin user creation feedback messages for clarity and actionability and formatted code

* Add fallback error message for admin creation failure in setup script

* Add run script for Uvicorn with dotenv integration

* Refactor server runner to use argparse for host and port configuration

* Remove captured output print statement from server runner

* Fix server runner to ensure cross-platform compatibility and improve log handling

* removed run.py to match original repo

* Fixing custom search not working properly

* Refactor search settings event listeners for improved functionality and clarity

* Update search function signatures to use Optional for count parameter

* revert changes

* fixed broken merge issue

* Delete services/chat_data_scraper.py

added by mistake

---------

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
2026-06-09 02:20:59 +01:00
pewdiepie-archdaemon af1aa34b4a Cookbook UI: Ollama browser, advanced serve fold, API tokens form, diagnosis toolbar, polish
Surface a lot of accumulated cookbook + UI work as a single non-agent
commit so the agent rework lands cleanly.

Highlights:
- Ollama as a first-class backend in the Cookbook:
  * Download input accepts ollama-style names (name:tag) → backend=ollama
  * /api/cookbook/ollama/library (cached scrape of ollama.com + curated
    fallback so classic models like qwen2.5 stay reachable)
  * "Browse Ollama library" toggle below Download with size chips
  * Engine=Ollama in hwfit toolbar merges the Ollama library into the
    main scan list as per-tag rows with the same Fit/Param/Quant/VRAM
    columns; click → fills Download input
- API Tokens form added to Integrations panel (matching wired
  loadTokens()/initTokenForm() that had no HTML)
- Serve panel polish: Advanced fold tightening (-8px nudges on vLLM
  checks, Extra args, Spec row), n_cpu_moe + Split Mode controls
  pulled up 8px to align with the row's checkboxes, GGUF File dropdown
  exposed for Ollama backend, GPU re-render on Edit serve restore,
  _forceBackend flag so saved serveState wins over backend detection,
  cookbook:servers-changed CustomEvent so panels don't need refresh
- Models page redesign: Add Models row (URL + hidden API key reveal +
  Type select + Scan/Ollama/Key/Test/Add icon buttons), Probe All +
  Clear-offline buttons in Added Models toolbar, offline-pill removed
  (opacity already conveys state), Engine dropdown gains Ollama option
- _ping_endpoint probes /v1/models then base, accepts 4xx as
  reachable (vLLM returns 404 on bare /v1, fully working endpoints
  were showing offline)
- Diagnosis card: × dismiss + Copy bundle buttons restored on the
  serve error feedback card
- Orphan tmux sweep re-enabled behind a 60s rate-limit + background
  Thread (off the main event loop) so dead serves get discovered
- cookbook_routes auto-register watchdog: drops the endpoint if the
  serve session exits non-zero within the first ~3min
- ollama-rocm sidecar awareness in download wrapper (`docker exec
  ollama-rocm ollama pull` when host ollama isn't installed)
- Skill extractor sets initial_status="published" when
  auto_approve_skills pref is on (audit demotes later)
- Skill list / model list / cookbook scan misc polish
2026-06-09 09:46:19 +09:00
CorVous 678b141063 fix(memory): make auto-memory extraction reliable for reasoning models (#3190)
* fix(memory): auto-memory extracted nothing — flatten window so the prompt ends on a user turn

extract_and_store appended the recent window as raw alternating role messages
after the system prompt. Since the window is the last N messages, the prompt
usually ENDED on an assistant turn — and a chat model given a prompt ending on
an assistant turn returns an empty completion (nothing to answer). The result
was facts=[] → "Auto memory extraction ran: 0 candidates" on every run, so no
memories were ever stored, while skill extraction (which flattens the transcript
into a single user message) worked fine.

Flatten the window into one user message ending with an explicit instruction,
mirroring the skill extractor, so the model always responds. Also harden parsing
for reasoning models, matching the audit path which already does this:
- raise max_tokens 500 → 4096 (a reasoning model spends the budget on <think>
  before emitting JSON; 500 truncated it before any JSON appeared);
- strip <think>/prose preambles via strip_think and slice the embedded JSON
  array before json.loads, instead of bombing on char 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: tighten memory-extraction-empty-completion — clarify JSON-slice comment re prior strip steps

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(memory): reframe the comment to the accurate root cause (raw-chat framing)

The earlier comment leaned on "ends on an assistant turn -> empty completion",
which is only one failure mode. The dominant cause, confirmed by a controlled
repro (0/6 old vs 6/6 new on this model), is that passing the window as raw chat
messages makes the model treat it as a conversation to continue rather than a
transcript to analyze, so it returns [] even when durable facts are present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(memory): cover extraction JSON parsing + slice trailing commentary unconditionally

Factor the strip/fence/slice/json.loads logic out of extract_and_store into
a pure module-level helper _parse_extraction_json(raw) -> list and drop the
'text[0] != "["' guard so the array is sliced whenever both brackets exist
(fixes trailing commentary like '[...] Done!' reaching json.loads).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:57:44 +02:00
Mike 525e3ac9bb refactor(constants): single source of truth for data dir (#3368)
* refactor(constants): single source of truth for data dir + merge core/src constants

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(contributing): use named src.constants for data paths, drop core/constants references

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 09:58:52 +02:00
Lucas Daniel ef394578db fix(search): catch HTTPStatusError so 403/404 URLs degrade gracefully instead of 500 (#2203)
raise_for_status() raises httpx.HTTPStatusError for 4xx/5xx responses,
but the surrounding try/except only caught httpx.RequestError (network
errors) and RateLimitError (429). Any other HTTP error code propagated
uncaught up through chat_processor -> chat_helpers -> chat_routes and
surfaced as a 500 Internal Server Error.

Added an explicit except httpx.HTTPStatusError clause that logs a warning
and returns an empty result, matching the behaviour already in place for
network errors.

Also adds focused regression tests that exercise the real
fetch_webpage_content() path with a mocked _get_public_url:
- 403/404 responses return the standard empty-result shape instead of
  raising, proving the new HTTPStatusError handling works end to end.
- 429 responses still take their own dedicated rate-limit branch (the
  status_code == 429 check runs before raise_for_status() is reached),
  keeping that behaviour distinct from the new generic HTTPStatusError
  handling.

Dropped the unrelated builtin_mcp.py change that had been carried over
from a rebase; that fix is tracked separately in #2018 and this branch
should stay scoped to the search content fetch path.

Closes #2148
2026-06-08 01:09:21 +01:00
horribleCodes 8eb4337aa1 fix(platform): Improve WSL SSH remote compatibility (#3316)
* fix(platform): add WSL compatibility functions and path translation
fix(cookbook): enhance model scan script to support additional HuggingFace cache paths
fix(hardware): improve cache key generation for remote SSH context
test(tests): add tests for WSL detection and path translation functionality

* fix(cookbook): prefer prebuilt wheels for llama-cpp-python and normalize package aliases

* fix: enable StrictHostKeyChecking in nvidia probe
refactor: consolidate ssh & powershell command execution to utility functions in core module
refactor: consolidate nvidia path candidates in to single variables in core module
tests: add tests for new utility functions

* fix: correct wrong variable name
2026-06-08 00:33:50 +02:00
Lucas Daniel 40cc1f6b7e fix(skill-extractor): walk all brace candidates so stray braces in prose do not swallow valid JSON (#2205)
* fix(skill-extractor): walk all brace candidates so stray braces in prose do not swallow valid JSON

The extractor sliced from the FIRST brace to the LAST brace to recover
JSON embedded in surrounding commentary. When the model emits stray
braces before the JSON object, the slice produces invalid JSON,
json.loads raises, and the exception is swallowed -- the skill is
silently lost.

Fix: walk each brace candidate left-to-right and attempt json.loads on
each slice. The first candidate that parses successfully wins. If none
parse, json.loads on the original text raises and the existing
JSONDecodeError handler logs and returns None as before.

Tested locally -- 8/8 tests passed:
  tests/test_extract_skill_json_nonstring.py (2 passed)
  tests/test_skill_extractor_rows.py (1 passed)
  tests/test_search_content_extraction_parity.py (2 passed)
  tests/test_deep_research_search_error.py (3 passed)

Closes #2199

* test(skill-extractor): add focused repro for stray-brace JSON recovery

* test(skill-extractor): add regression test for leading invalid-brace fragment

Addresses the remaining edge case from review: a response that *starts*
with a brace but the leading fragment isn't valid JSON (e.g.
'{not json}\n{"title": "Valid later", ...}') still needs to recover
the valid skill object that follows.

_extract_json_object (already on dev) handles this correctly — it tries
the whole de-fenced string first, then walks each '{' candidate left-to-
right regardless of whether the response begins with '{', so the leading
invalid fragment no longer short-circuits recovery of the real object.
Updates the comment at the call site to call this out explicitly and adds
a regression test covering exactly the scenario described in review.
2026-06-07 23:31:12 +01:00
Kenny Van de Maele d6003ee885 fix(search): write cache under DATA_DIR, guard mkdir against read-only path (#3334)
services/search/cache.py set CACHE_DIR = services/cache (the source tree) and
mkdir'd it at import, unguarded. In Docker services/ is the read-only image
layer, so the mkdir fails at import (same class as the analytics bug #2366).
Move the cache under DATA_DIR/cache (writable on Docker and native) and wrap
the mkdir so an unwritable path disables disk cache instead of crashing import.

Part of #3331.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 22:37:12 +01:00
Giuseppe Castelluccio 574d65b8a7 fix: search analytics FileHandler crashes on startup writing to read-only image layer (#2366)
* fix: move search analytics log to writable /app/logs volume

services/search/analytics.py opened a FileHandler at module import
time pointing to /app/services/search_engine_error.log — inside the
container image's read-only layer. The process runs as non-root so
the open() fails with PermissionError, crashing uvicorn before it
ever binds. ANALYTICS_FILE had the same problem.

Both paths now point to /app/logs (bind-mounted from the host data
directory). The FileHandler creation is wrapped in try/except so a
missing mount doesn't hard-crash on import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: derive log dir from DATA_DIR instead of hardcoded /app/logs

Fixes reviewer feedback on #2366: /app/logs only exists inside Docker,
so native runs couldn't write the analytics file. DATA_DIR resolves to
the repo's data/ directory on native and /app/data (writable mount) in
Docker, making both the error log handler and ANALYTICS_FILE work on
every platform.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 19:26:22 +02:00
Bipin Mishra d3f107cba6 fix(hwfit): detect NVIDIA GPU on WSL and other minimal-PATH environments (#3306)
The nvidia-smi absolute-path fallback in _detect_nvidia() was gated
on _remote_host, so it never ran for local detection. On systems
where nvidia-smi is not in the default PATH (e.g. WSL: /usr/lib/wsl/lib/),
this caused the Cookbook to report 'No GPU' even when nvidia-smi works
from an interactive shell.

Two issues fixed:
1. Removed the _remote_host gate so the absolute-path scan runs for
   local detection too.
2. For local execution, pass arguments as a list instead of a string
   so subprocess.run() resolves the absolute path correctly. Remote
   (SSH) execution keeps the string form, which the SSH command builder
   handles.

Co-authored-by: Bipin Mishra <bipin.mishra@atlascopco.com>
2026-06-07 17:53:49 +02:00
Mazen Tamer Salah 4974cf00f5 fix(skills): tolerate a stray brace before the JSON in skill extraction (#2200)
maybe_extract_skill() sliced the LLM response from the first '{' to the
last '}'. When a model emits a stray brace in prose before the real
object (e.g. "uses {placeholder} then {...}"), the slice starts at the
prose brace, json.loads fails, and a valid skill is silently dropped.

Factor parsing into _extract_json_object(), which tries the whole
(de-fenced) string first and then each '{' start position, returning the
first candidate that parses to a JSON object.

Adds tests/test_skill_extractor_json.py.
2026-06-07 16:54:36 +02:00
SurprisedDuck 3353a4e97e fix(memory): record dislikes as dislikes, not preferences (#2435)
_fallback_memory_candidates matched both positive (prefer/like/love) and
negative (hate / do not like / don't like) sentiment verbs in one regex
alternation, then formatted every hit as "User prefers {X}.". So
"I hate cilantro" was stored as "User prefers cilantro." -- the inverse of
what the user said. These fallback facts are persisted to memory and later
re-injected into the model's context, so the inverted preference actively
misleads the assistant.

Capture the matched verb and branch on it: negatives become
"User dislikes {X}.", positives stay "User prefers {X}." (still filed under
the existing "preference" category).

Supported by Claude Opus 4.8

Co-authored-by: SurprisedDuck <288741682+SurprisedDuck@users.noreply.github.com>
2026-06-07 16:36:07 +02:00
n2b12 e852c1df4f VRAM detection under native Windows install (#1610)
* Convert to different style of comment to make it easier to work with, fix formatting inside Powershell script.

* Grab VRAM amount from driver's registry keys.

* Fixed regression on NVIDIA GPUs
2026-06-05 22:49:47 +02:00
horribleCodes ba657fd078 fix: Add WSL paths to hardware detection fallback (#2933)
This change extends both the `PATH` variable and the list of absolute paths used to locate the `nvidia-smi` package to include `/usr/lib/wsl/lib`.
This path is a candidate for the default location of nvidia-smi for WSL machines (tested on WSL Ubuntu 22.04.5).
2026-06-05 21:34:41 +02:00