65 Commits
Author SHA1 Message Date
Joshua MoonandCodex 19356e1a61 fix(search): retain filtering and rank evidence
Filter identifier occurrences by kind before choosing each file representative. Keep the strongest lane evidence in the representative used for reranking while preserving identifier file ordering and reciprocal-rank weights.

Document the snippet scan bound and distinguish historical ranking captures from results of the corrected evaluation runner.

Validated with 593 default tests, 617 CUDA-enabled tests, and 60 evaluation tests. Regression coverage includes later eligible occurrences and a fused caller matching identifier and phrase lanes.

Co-authored-by: Codex <[email protected]>
2026-09-30 11:50:33 -04:00
Joshua MoonandCodex fa3d62faad fix(cli): make paging and filters executable
Keep next-page command payloads free of advisory prose and emit page-size or budget advice on a separate hint line. Normalize literal percent characters and valid path escapes consistently. Document and test quoting the full negative-filter query without relaxing unknown-option rejection.

Validated by paging continuation, percent-path, quoted argument, and strict-option regressions in the 593 default and 617 CUDA-enabled test suites.

Co-authored-by: Codex <[email protected]>
2026-09-30 11:50:11 -04:00
Joshua MoonandCodex 42f3e57bc1 fix(search): preserve worktree lookup evidence
Keep container identity during stale symbol re-extraction. Apply exact owner matching and reference priority before result caps. Disclose bounded or incomplete worktree scans and prevent incomplete home misses from widening; refresh exact misses to discover newly added paths.

Migrate the shared re-extraction helper and callers together. Retain explicit unverified parent bytes when an unextractable changed file still has a usable candidate.

Validated with 593 default and 617 CUDA-enabled tests, including swapped methods, crowded qualified owners, declaration priority, and worktree coverage regressions.

Co-authored-by: Codex <[email protected]>
2026-09-30 11:49:48 -04:00
Joshua MoonandCodex 8a26e31b63 fix(runtime): preserve deadlines and stop bounds
Carry the accepted query deadline through local dispatch and router forwarding, and include spool queue time. Stop joining blocked request workers after the drain window and reject retained queued jobs.

Validated with 593 default and 617 CUDA-enabled tests, including expired dispatch/router deadlines and blocked shutdown regressions.

Co-authored-by: Codex <[email protected]>
2026-09-30 11:49:15 -04:00
Joshua MoonandClaude Opus 5.5 2f0598145a Merge branch 'wave/b' into wave/integration
Concurrent daemons, retry policy, stale sym: fallback, warming reads,
and the shared tokenizer (lane B, ee503ec). Reconciliations:
- src/cli/emission.rs: lane C's `help` word arm stays first; the
  execution arm now runs through retry::run_with_retry.
- src/source/acquisition.rs + acquisition/symbol.rs: lane D's symbol
  split wins; AcquiredSource keeps D's SymbolSelection and adds B's
  `freshness`; acquire_symbol's read_first closure calls B's
  stale_symbol::acquire_symbol_entry (the republish wait is skipped for
  parent views, so lane A's parent_show read_symbol path is unchanged).
  parent_show.rs's two AcquiredSource literals gain `freshness: None`;
  the stale-symbol test uses D's acquire(.., InvocationDirectory).
- src/source.rs: `freshness` renders before D's definitions/imports block.
- Lane A tests that open WorkspaceResults (home_index/tests.rs,
  result_cache/tests.rs) create the store first, as B's create/open
  split requires; production callers (coordinator start, local runs)
  already create it.
- docs/index-contract.md: A's worktree read-path text kept; B's stale
  sym: and warming-read rules condensed after it (file stays <=200 lines).
Worktree warming in workspace mode still answers from the parent index
(A's resolve_home_index, which never goes through a root daemon);
B's index_warming applies to a root's own unpublished index.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 10:12:16 -04:00
Joshua MoonandClaude Opus 5.5 da02f13a58 Merge branch 'wave/a' into wave/integration
Lane A (warming worktrees answer from the parent member's index,
per-hit worktree verification, coverage honesty, silent seeding)
joined with lanes C, D, E and F.

Reconciliations:
- Ranking: search::rank_definitions(hits, origin) is the single entry
  point (path order, then D's declaration tier, non-test, invocation
  proximity); definitions(), exact sym: search, the workspace sym:
  re-rank and worktree re-extraction all use it. rank_declarations is
  internal to search::declarations.
- show sym: through a parent index uses D's read_symbol (split out of
  acquire_symbol): import following, SymbolSelection counts and also:
  locators apply to worktree-ranked candidates; the shown row falls
  back to worktree re-extraction or unverified index bytes on
  stale_source.
- Paths: worktree re-extraction filters changed files with C's bound
  PathFilter (include/exclude) and matches `A::b` via
  search::names_definition (D's QualifiedName).
- map: a changed file's re-extracted outline uses D's selection and
  folding (structure_map::outline_extracted, sharing OUTLINE_KINDS and
  FOLDED_KINDS with the SQL); no duplicate kind list in lane A.
- Lines pages: C's page_text/more_command footer wrapped by A's
  mark_worktree_differs; A's coverage wording, C's filter diagnosis,
  and D's refs summary all render on the coverage line.
- Test fixtures gain Hit.repeats and MemberSnapshot.parent_index.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 10:07:03 -04:00
Joshua MoonandClaude Opus 5.5 ee503ec971 fix(cli): never wait on the writer lease to read
A client that saw no daemon reconciled in the request with a blocking
writer lease, so a daemon that started meanwhile made it wait out the
whole initial scan. Without --no-daemon the in-request reconcile now
uses Store::index_unless_leased and leaves the index warming when the
lease is held.

Warming handle, more, ctx, and path reads no longer open the writer
(DDL, INSERT, 30 s busy timeout behind a publication): they use
Store::unpublished, an empty in-memory schema at generation 0 that
saves result sets in the cache and writes nothing to index.sqlite3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 10:06:26 -04:00
Joshua MoonandClaude Opus 5.5 0914c8e57d Merge branch 'wave/e' into wave/integration
Lane E concept ranking: evidence lanes (qualified definitions,
identifier occurrences, phrase, all-terms), identifier evidence tiers,
the 0.7 path prior, rerank tiers, snippet choice, and the
concept-audit evaluation manifest.

Conversions made while merging:
- search_prepared runs lane E's concept lanes with lane C's bound
  path filter: concept_lanes, the occurrence lane (its raw
  substr(f.path,...) check is now paths.sql_clause("f.path")),
  lexical/file lanes and semantic_lane::append all take `paths`.
- PathPrior::for_query reads query.path.includes() for test/doc
  component checks instead of a raw prefix string.
- Lane D's declarations::is_test_path is deleted; declarations and
  reference_sites call path_class::is_test_path, so fixtures/, bare
  `test` stems and generic *_test stems are production paths there
  too (D's classifier test moves evaluation/fixtures/x.dm to the
  negatives).
- The A::b concept lane calls D's declarations::qualified_hits
  (QualifiedName::matches) instead of lane E's own container matcher.
- The file-ranking section moves from index-contract.md (then over
  200 lines) to a new docs/ranking-contract.md; links follow.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:59:49 -04:00
Joshua MoonandClaude Opus 5.5 bca2bbd140 fix(daemon): survive panics and bound nested waits
Review follow-ups for concurrent serving:
- Pool workers catch job panics; a panicking request answers
  `internal_error: MESSAGE` (sockets and spool) and the worker keeps
  serving. A panic in reconcile or idle work fails the server, which
  releases its socket and lock so the next client starts a fresh daemon.
- A `daemon_busy` refusal reads the request (200 ms) before replying, so
  a client still writing never sees EPIPE; retry.rs also treats EPIPE and
  ECONNRESET on read verbs as busy.
- Socket timeouts are retried only when they struck within 5 s; a full
  30 s (or relayed 28 s) wait is final, so a retry never doubles it.
- A router `daemon_unavailable` reply falls back to direct execution;
  other router errors remain the answer.
- daemon::running probes with a socket connect instead of the startup
  flock, which could make a starting daemon exit; ensure_member uses it.
  A connection closed before any request gets no reply.
- Nested waits are deadline-bound: run_direct takes a reply wait (the
  coordinator passes its query deadline's remainder), and the router's
  spawn wait plus forwarded read share one 28 s budget (request_by).
- A spool client whose claiming router stops beating fails at once with
  `daemon_unavailable`; stop lets accepted requests finish (up to 28 s)
  after releasing the socket.
- src/daemon/server/tests.rs is split; failure tests live in
  server/tests/failures.rs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:57:22 -04:00
Joshua MoonandClaude Opus 5.5 09e537a87c fix(workspace): bound worktree hash checks
- WorktreeHashes is one query's hash cache: each file is hashed at
  most once and at most 128 files in all. worktree_symbols (sym:
  search and show sym:) and check_hits share it, so a common name no
  longer hashes every parent-hit file; parent hits past the budget
  stay unverified and are flagged from the divergence as elsewhere.
- Files an answer re-read (re-extracted, gone, unextractable, or a
  re-extracted map file) count as differing in the footer, so the
  lagging-parent sym: case reads "(1 file differs)".
- show sym: runs worktree_symbols whenever any file diverges (or the
  divergence is incomplete), not only on stale_source/no_definition,
  so it agrees with sym: search on worktree-only redefinitions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:48:25 -04:00
Joshua MoonandClaude Opus 5.5 dfbdbdec3f fix(search): narrow test-path conventions
A bare `test` stem (Test.java, test.config.js), a crate named test
(library/test/src), generic `*_test` stems (ab_test.rs, speed_test.rs)
and fixtures/ directories are production paths. Test paths are now a
tests/, __tests__/, spec/ or testdata/ directory, a test/ directory
not followed by src/, a tests or *_tests stem, .test./.spec. names,
and the language conventions *_test.go, *_test.py, test_*.py and
*_spec.rb. docs/index-contract.md states the rule; lane D's ranking
can call path_class::is_test_path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:48:12 -04:00
Joshua MoonandClaude Opus 5.5 ad4963066c fix(search): demote tests and docs by 0.7, not 0.5
A 0.5 multiplier made a demoted file's best lane rank tie an undemoted
rank 61 and pushed test-file labels of the development tasks off the
first page (twelve-task R@10 0.9167 -> 0.8333). At 0.7 a demoted rank
r ties roughly 1.43r + 26; the concept-audit results are unchanged
(R@5 0.9618, MRR 0.935) and the twelve-task set has no per-task
regression (R@5 0.75 -> 0.9167, R@10 0.9167 -> 0.9167).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:45:25 -04:00
Joshua MoonandClaude Opus 5.5 32728601fe fix(workspace): verify parent hits against bytes
Git divergence misses a parent index that lags its HEAD and edits
inside the 5 s reuse window, leaving stale spans unflagged and
show sym: failing stale_source.

- hit_verification.rs: each distinct file among a parent answer's
  hits (up to 128) is hashed once and compared with the hit's
  indexed revision; mismatches set `differs` and count toward the
  footer, which lists divergent plus mismatched files (null only
  when divergence is incomplete). Later files fall back to the
  divergence, flagged when it is incomplete. Hits read from current
  bytes (live_regex, re-extraction) are never flagged.
- worktree_symbols: sym: search and show sym: re-extract up to 64
  changed files (divergent files first, then parent-hit files whose
  bytes mismatch), replace those files' parent hits with all current
  definitions of the name, add worktree-only redefinitions even when
  the parent has the name, and order by search::rank_definitions.
  Unextractable files keep parent hits; show serves their indexed
  bytes unverified, and a stale re-extracted handle answers
  stale_source instead of a missing-row error.
- Unavailable reasons carrying timed_out anywhere in the chain report
  the member as timed out.
- live_regex excludes both the index cache and the store's results
  cache (the worktree's own cache in a parent view).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:43:05 -04:00
Joshua MoonandClaude Opus 5.5 4a1fdc15a9 fix(source): keep stale_source when bytes won't parse
A stale sym: read whose current bytes are binary or cannot be extracted
(reextract_definition's source_excluded / extraction_unavailable) now
answers stale_source with that reason appended instead of replacing it.
Docs state which reads need a publication and which reconcile in the
request. The owner-status test bound rises to 20 s: under a loaded test
machine it took 13 s, still well short of the 28 s re-entry deadlock it
guards against.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:43:01 -04:00
Joshua MoonandClaude Opus 5.5 51fff475cd Merge branch 'wave/phase0' into wave/b
Brings in the query-deadline pause for in-request reconciles. The
dispatch conflict resolves to read_store taking the parsed options and a
&mut QueryDeadline: only a client that no daemon serves (or --no-daemon)
reconciles in the request, paused off the deadline; otherwise an
unpublished index answers index_warming for search/refs/map and sym:
reads, while more, ctx, handle, and path reads open the index without a
publication. The new workspace deadline test creates its result store
before opening it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:43:01 -04:00
Joshua MoonandClaude Opus 5.5 7f1d0ceab0 feat(daemon): serve requests concurrently
Every daemon tier (router, coordinator, root) now accepts on one thread
and answers on a bounded RequestPool (src/daemon/pool.rs: 16/8/4
workers, 64 queued); a full queue replies `daemon_busy: retry`.
Reconciliation, watching, root-gone detection, spool draining, and idle
work move to one maintenance thread (src/daemon/reconciler.rs), which
lowers its own priority only after the initial reconcile. Handlers
implement DaemonHandler (src/daemon/server.rs); each request's query
deadline starts at accept. The root daemon (src/cli/root_daemon.rs)
binds before any database work and answers `index_warming` for reads
until the first publication (explicit path reads still serve current
files); status reports `state: warming` without writing. Spooled
requests are claimed by rename to `<id>.claimed`.

Re-entry: workspace owner verbs and member semantic commands run via
cli::run_direct, never through the router still proxying the request
(the bare-status deadlock), and a router error is now the answer
instead of silently re-running the command directly. The router proxies
with a 28 s read timeout; clients wait 30 s (CLIENT_REPLY_WAIT, shared
with the spool) and retry read verbs once on EAGAIN/TimedOut,
daemon_busy, database is locked, or (after 2 s) index_warming
(src/cli/retry.rs).

Coordinators build a SemanticSession per request, create
workspace.sqlite3 once (WorkspaceResults::create/open), and never run
workspace index/init; `ws status` opens members query-only. Spawned
daemons are reaped by their own waiter thread (BackgroundChild) instead
of a racing waitpid(-1), and a losing startup race keeps polling the
winner. DiagnosticQueue is Sync so concurrent requests share it.

A `sym:` read whose file changed waits up to 3 s for a serving daemon to
republish, then re-extracts the definition from current bytes and
answers `verified: false, freshness: unpublished`; handle reads keep
`stale_source`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:38:13 -04:00
Joshua MoonandClaude Opus 5.5 fdf9497a9d fix(search): tier named files, apply prior once
Review fixes for concept ranking.

- Identifier queries rank in evidence tiers: a file whose stem is the
  token (tick_rate.rs), then declaring files, then files using, binding
  or importing it, then the rest. Exact ×3 applies to declarations only,
  not locals, parameters or imports.
- The path prior multiplies each lane contribution once: lexical lanes
  in the first fusion, semantic ranks in the re-fusion (was ×0.25 with
  semantic). kind:file queries and, outside identifier queries, files
  whose basename holds every term are never demoted; DOC_TERMS gains
  md, agents, claude, guide; file: filters match path components.
- path_class: add spec/ and *_spec, drop fixtures/ (content packs keep
  production code there).
- A::b adds a qualified definition lane (definitions of b whose
  container or module path names A); the qualified occurrence lane uses
  an IN subquery instead of a correlated EXISTS.
- Rerank tiers follow the fusion tiers and apply identifier evidence
  only to identifier queries, so plain words are not pinned.
- Snippets prefer code unless a comment has two more terms; `"a" b "c"`
  is no longer one literal.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:36:10 -04:00
Joshua MoonandClaude Opus 5.5 200ddfa8bc fix(workspace): read index bytes when unextractable
reextract_definition now reports source_excluded (binary) and
extraction_unavailable (cancelled or failed extraction). A parent-view
show whose worktree file hits either no longer fails: it serves the
parent index's stored bytes for the hit, verified: false, with
source: parent_index and served_from "MEMBER index; worktree file not
extractable" (ParentRead::Indexed). A handle read and a sym: read both
fall back this way; a file that parses without the definition still
answers stale_source/no_definition. map of such a file keeps the
parent's rows, flagged differs.

show code moves to home_index/parent_show.rs; render_owned lets
metadata name the source of unverified bytes instead of assuming
current_file; source::usable_extraction is shared with map.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:34:36 -04:00
Joshua MoonandClaude Opus 5.5 8cdd6f1ecd Merge branch 'wave/d' into wave/integration
Converts lane D's raw substr(f.path,...) prefix checks in
declarations::qualified_hits, reference_sites::references and the
structure_map directory query to BoundPathFilter::sql_clause, and makes
map's single-file mode use PathFilter::resolve_single_file; keeps
Phase 0's open_read/QueryDeadline in workspace show and C's
filter_diagnosis coverage part beside the refs summary.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:33:06 -04:00
Joshua MoonandClaude Opus 5.5 d7dbfcabb1 Merge wave/phase0 deadline fix; address review items
Merges 6260bfa (reconciles off the query deadline). --no-daemon search
now indexes each member under deadline.pause_during before
resolve_home_index opens it; HomeIndexPolicy::IndexInline only reads.

Phase 0 review fixes for lane A:
- Reads never open the writer: navigation's warming fallback and
  MemberSnapshot::open's generation-0 fallback to Store::open are
  gone. A show with no published own or parent index answers
  index_warming; --no-daemon show indexes the member first, paused
  off the deadline.
- The parent view requires linked_root_of_member (shared Git common
  directory) before Store::reading_from.
- search_prepared gets the answering index's cache
  (store.index_cache()), so semantic vectors and live-regex cache
  exclusion match the parent index in a fallback.
- Worktree caches record a cache-root marker; the sweep evicts a
  cache holding only the marker (e.g. results.sqlite3 from parent
  reads) once its root is gone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:29:23 -04:00
Joshua Moon 9048c91d35 Merge branch 'wave/c' into wave/integration 2026-09-30 09:28:07 -04:00
Joshua MoonandClaude Opus 5.5 9040ab7507 fix(search): bind path filters once and anchor safely
Review follow-ups for lane C:

- The semantic lane uses the shared path filter, so `--features
  semantic` compiles again.
- `search_prepared` binds the `PathFilter` once and passes the bound
  filter to the exact, lexical, file, semantic, and live lanes and to
  zero-hit diagnosis. Bound filters render inline SQL: nothing when
  empty, `path>=lo AND path<hi` index ranges for root prefixes, and
  `instr` for component matches.
- Root anchoring requires an indexed path continuing the value at `/`,
  `.`, or its end, so `file:src` beside `src-tauri/` still reaches
  `crates/a/src/`.
- `file:` values are percent-encoded like indexed paths (`%` kept).
- `PathFilter::resolve_single_file` lets `map ./src/x.rs` and
  `map host.rs` list members; directory prefixes never do.
- Diagnosis counts in SQL and reads nearest-path candidates only for
  values that matched nothing, ranked by relation and edit distance.
- Usage errors are also written to stderr; an empty `lang:` filters
  nothing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:27:31 -04:00
Joshua MoonandClaude Opus 5.5 6d9e0113d4 fix(search): tighten lane D navigation per review
- map: extern block items (including Rust 2024 `unsafe extern`, which
  the pinned grammar parses as ERROR + block) carry the exact
  `extern block` container; map filters on it instead of a LIKE prefix
  that hid `mod externals` and `class ExternalApi` members.
- refs: collapse only same path, line, role and resolution; the count
  lives in a new `Hit.repeats` field (JSON `repeats`, lines `×N`), so
  `resolution` stays an enum. Test sites rank last whatever their role.
  Qualified refs pre-filter owners in SQL.
- show: import trails follow up to four hops with a visited set,
  resolve crate/self/super against the importing module (crate root =
  path before the last `src/`, plus enclosing inline mods), report the
  chain, and mark ties as `(1 of N)` with the tied declarations first
  in `also:`. Import tracing moves to search/import_trail.rs.
- Qualified lookups keep the pre-filter truncation flag.
- Workspace `search 'sym:'` ranks by the invocation directory; the
  directory is compared percent-encoded. Test scopes match any
  `tests` container segment.
- Workspace map misses count as no home hits; `no_indexed_path` is
  returned only when no member has a row.
- acquisition.rs splits into acquisition/{symbol,tests}.rs.
- Navigation contract moves to docs/navigation-contract.md with the
  JSON fields; language contract documents the impl-container limit.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:26:37 -04:00
Joshua MoonandClaude Opus 5.5 2942d2b005 feat(workspace): fall back to parent index
A linked worktree whose own index is unpublished now answers at once
from its member's published index, opened query-only and reading the
worktree's current bytes (src/workspace/home_index.rs,
resolve_home_index -> Own | Parent | Warming | Unavailable). Search,
refs, map, and both show paths use it; --no-daemon still indexes
inline, and a home with neither index waits up to 8 s as before.

- Divergence (home_index/worktree_divergence.rs): worktree diff
  against the parent's HEAD, worktree untracked files, and the
  parent's own uncommitted and untracked files, each Git probe bounded
  to 2 s (history::git::run_bounded) and reused from
  divergence.json for 5 s. Hits in those files carry `differs`
  (JSON) and a trailing `differs` field (lines).
- Changed files: show re-extracts a definition in a changed file,
  sym: with no hit re-extracts up to 64 changed files, and map FILE
  re-extracts a changed file (served_from: "MEMBER index;
  re-extracted in worktree"). A stale parent-view handle re-reads
  the same-name/kind definition from current bytes.
- Handles: MemberSnapshot.parent_index records the parent index, so
  show HANDLE reopens the same view after the worktree publishes.
- Coverage: state parent_fallback with home_state, served_from,
  differs, differing_hits; footer "M@WT warming -> served from M
  index (N files differ)". Warming/unavailable homes keep a reason
  and never widen: scope "home (warming; ws:all searches N members)"
  on an empty page. Partial coverage names its kind ("partial (20
  unsearched: fact_limit)", issues.unsearched_paths); live regex
  counts binary and oversized files as excluded, not unsearched.
- Seeding is silent: outcomes go to seed-outcome.json, shown by
  ws status.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:23:49 -04:00
Joshua MoonandClaude Opus 5.5 994d33fe84 refactor(search): share path classes, keep lanes additive
Move test/doc path classification into src/search/path_class.rs
(`is_test_path`, `is_doc_pointer_path`). Its test rule is a superset of
lane D's declarations::is_test_path (tests/test/__tests__/fixtures/
testdata directories, tests/test stems, test_*, *_test, *_tests,
.spec., .test.), matched ASCII case-insensitively, so both rankers can
use one classifier.

Restore lexical_hits and fts_terms to their master form: the phrase
and all-terms lanes relabel lexical hits in concept_query instead of
threading a provenance parameter, and the fused entry point keeps the
fuse_search_file_lanes name, so merges with lanes C and D stay
mechanical.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:14:02 -04:00
Joshua MoonandClaude Opus 5.5 6260bfa54b fix(store): keep reconciles off the query deadline
In-request reconciliation no longer spends a query's 20 s budget:
QueryDeadline::pause_during moves the deadline by the reconcile's
duration, used by dispatch's read_store and by each --no-daemon
member reconcile in workspace retrieval, so a slow first member no
longer times out later members.

Workspace show/ctx and history reads classify SQLite interrupts as
timed_out. An all-failed workspace search answers timed_out only when
some member actually timed out. reextract_definition reads with the
index's 2 MiB limit and reports binary files and cancelled or
factless extractions as errors instead of None.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:10:57 -04:00
Joshua MoonandClaude Opus 5.5 352e9f182e feat(search): rank concept queries by evidence
Free-text search now runs evidence lanes in fusion order: exact
definitions, identifier occurrences, phrase and all-terms FTS5 lanes,
then the any-term and filename lanes.

- A single identifier-shaped token (_, ::, camelCase, digits) weights
  exact definitions x3, adds an occurrence lane, and ranks files that
  define or use the token ahead of files lacking it.
- Two or more terms add a phrase lane (x2) and an all-terms lane; a
  "quoted" literal runs the phrase lane alone and skips semantic.
- A deterministic path prior halves test and Markdown paths unless the
  query names tests/docs, sets lang:text, or file: targets them.
- Rerank keeps phrase/identifier hits ahead of other hits and orders
  within each tier.
- Previews pick the span line with the most distinct query terms.

Contracts: docs/index-contract.md (File ranking), retrieval, semantic
rerank, and the cli.md snippet sentence.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:10:14 -04:00
Joshua MoonandClaude Opus 5.5 e0410c76e5 feat(search): rank and follow definitions precisely
show 'sym:' ranks namesakes by kind tier, then non-test before test
sites, then nearest the invocation directory (workspace cwd relative
to the member root), then path. The footer counts declarations apart
from imports (`definitions: N (+M imports)`). An import that outranks
every declaration is followed through its `use` spelling to the
original (`via: re-export of a::b at PATH:LINE`) or reported as not
indexed. `sym:A::b` matches `b` owned by impl/trait/class `A` or under
module path `A`, in search and show. Every word after `show sym:` and
`refs NAME` joins the query, so unquoted file:/lang:/kind: apply;
no_definition names active filters and the best unfiltered match.

refs honors file:/lang:/kind: (role or target kind) and qualified
names, orders declaration, code, imports, then tests, collapses one
path:line into a `×N` row, and reports `refs T sites in F files` in
single-repo and workspace coverage.

map FILE lists code items before fields, variants and mod rows and
drops the whole-file module and extern signatures (extern items now
record their block header as container; extraction revision 6).
map PREFIX gives one row per file naming its top-level items; an
unknown path fails with no_indexed_path and nearest paths.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 09:03:06 -04:00
Joshua MoonandClaude Opus 5.5 ae95adebe4 feat(store): add query-only read path
Store::open_read opens index.sqlite3 read-only (query_only, no DDL,
2 s busy timeout capped by the query deadline, 2 MiB page cache) and
answers index_warming for a missing index or generation 0. A SQLite
progress handler interrupts statements once the request's
QueryDeadline (src/daemon/deadline.rs, 20 s) expires; classify()
relabels the interrupt as timed_out. reading_from() lets a worktree
read its own bytes through a parent member's index and save results
in its own cache. Store::index refuses on a reader.

Result sets move to <cache>/results.sqlite3 (ResultSetStore,
Store::result_sets), so saving results never queues behind a
publication; the old table is dropped at the next publish.

Search/show/refs/map/more/ctx in dispatch, workspace retrieval,
MemberSnapshot::open and navigation::read use open_read. The
workspace member loop takes a QueryDeadline and records members
reached after it as timed_out ("MEMBER timed out").

source::reextract_definition extracts one file in memory and returns
the same-name/kind definition nearest a line with its current
revision, for stale-read and worktree-diff fallbacks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 08:57:21 -04:00
Joshua MoonandClaude Opus 5.5 1346044d3c feat(search): add path filters and paging hints
Query surface fixes from the lunatic transcript audit:

- `file:` values OR when repeated and `-file:` excludes. A value matches
  root-relative prefixes, else any path-component start
  (`file:script/host` finds `crates/a/src/script/host.rs`). One
  `PathFilter` (src/search/path_filter.rs) binds against the index and
  compiles to a SQL predicate plus a Rust matcher shared by the exact,
  lexical, and file lanes, `map`, and live regex.
- Unknown `lang:` values fail with `unknown_language` and list the
  recorded languages; `md`/`markdown`/`toml`/`json` alias `text`.
- Empty filtered searches add `filter_diagnosis` to coverage: per-value
  matched path counts, nearest paths by name, `lang:X matched 0 files`,
  and combined or exclusion failures. Workspace pages group them by
  member.
- Lines footers print the command: `next: more CURSOR` (first page adds
  the `-n` or `-b` widening hint) and `next: show CURSOR`. JSON keeps
  the bare cursor.
- `next`/`page` verbs, bare cursors, and handles get targeted
  `unknown_command` hints; `help` prints help; clap usage errors are
  plain unbudgeted text with a `more` tip for paging flags.
- `--help` lists `-file:`, `-n`, `-b`, and the `more` footer usage.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-09-30 08:54:37 -04:00
Joshua MoonandCodex 2a38fe7b5f feat(php): resolve XFCP chains
Add bounded ordered XFCP predecessor candidates and resolve literal parent calls through concrete PHP methods. Publish typed hierarchy edges, preserve priority ties as candidates, and version resolver projections independently from extraction caches.

Validation: 471 Rust tests, release build, 58 Python plugin tests, and XenForo acceptance across 7,550 PHP files (99.1% complete, 57 CLI calls, zero failures, 10 XFCP occurrences, 8 parent calls, 4 Forum tie pairs, 25 User tie pairs).

Co-authored-by: Codex <[email protected]>
2026-09-28 18:23:03 -04:00
Joshua MoonandCodex 708b527e7b feat(php): index symbols and XenForo add-ons
Index PHP and PHTML declarations and references with conservative fully
qualified candidate lookup. Derive add-on dependencies from XenForo
manifests and class-extension metadata while leaving runtime XFCP chains
unresolved.

Add cache invalidation, language filters, agent steering, documentation,
and a read-only XenForo acceptance runner with isolated test daemons.

Validation: cargo test --locked (409 passed); Python plugin suite (58
passed); release XenForo acceptance (8 CLI calls, 7550 PHP files, 7482
complete, 13 cross-add-on candidate edges); scoped rustfmt and diff-check.

Co-authored-by: Codex <[email protected]>
2026-09-28 15:36:30 -04:00
Joshua MoonandCodex 3ef5b03757 feat(search): add C# and Rustdoc navigation
Add C# structural extraction and language filters, JavaScript member-function symbols, extraction cache invalidation, and optional Rustdoc navigation in the agent skill. Preserve conservative binding and relationship evidence.

Validation: cargo test --locked (391 passed); Python plugin suite (57 passed); git diff --check.

Co-authored-by: Codex <[email protected]>
2026-09-28 14:34:47 -04:00
Joshua MoonandClaude Opus 5.5 1b722c6a95 fix(search): rank sym: declarations before imports
`sym:` pages were in path order, so a widely imported type showed a
page of `use` lines before its definition (sym:TileCoord in lunatic
listed 18 imports first). Exact results now order declarations, then
modules, members, locals, and imports, keeping path order within each.

Validation: cargo test passes (379).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:53:20 -04:00
Joshua MoonandClaude Opus 5.5 d719c292bb perf(daemon): stop idle daemons re-walking trees
Watching daemons now fall back to a periodic reconcile every five
minutes instead of every 30 seconds, ignore watch events that only touch
.git, target, node_modules, or .trufflepig, and run at nice 10 with idle
I/O priority. Polling daemons keep the 30-second interval.

On this machine ~40 idle daemons averaged about one core in total over
three days (tgstation 29%, lunatic 20% of a core each), and agents
blamed trufflepig indexers for slow builds.

Validation: cargo test passes (378).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:48:04 -04:00
Joshua MoonandClaude Opus 5.5 f48afdae9a feat(source): clearer show header and read budget
Lines output of `show` now opens with `PATH [(MEMBER)] lines FIRST-LAST`
instead of a 64-hex revision and byte offsets, and explicit path reads
end with `verified: current file` (JSON `"source": "current_file"`)
rather than a bare `verified: false` that read as a failure. `show`
defaults to a 1500-token budget, never below a workspace page budget,
so a typical definition body fits in one read.

Validation: cargo test passes (377); plugin, evaluation, and navigation
replay checks pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:46:05 -04:00
Joshua MoonandClaude Opus 5.5 fc6d988b35 feat(source): read a definition body with show sym:
`show 'sym:NAME [file:P] [lang:L] [kind:K]'` reads the best-ranked
definition with that exact name as a verified handle read: declarations
before modules, members, then locals and imports. The footer reports
`definitions: N` and up to five `also:` path locators, valid in single
repositories and workspaces alike; a missing name fails with
`no_definition`. `map` of a single file also lists its functions,
methods, constants, and macros.

Transcripts showed ~3k `grep -n "fn X" -A30` body reads and ~1k
single-file outline greps per three weeks; each is now one call.

Validation: cargo test passes (377).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:43:17 -04:00
Joshua MoonandClaude Opus 5.5 9566c866cf fix(workspace): report only real coverage gaps
A member is partial only when files could not be examined (walk,
live-read, truncation), shown as `partial (N unsearched)`. Excluded
binaries, parse failures, and semantic/rerank status stay in `issues`;
lanes whose reason says they were never configured drop out of the
lines summary. The scope note uses parentheses so the summary's `; `
separators stay unambiguous, and the audit parser records it instead
of treating it as a member.

Every transcripted search reported `partial`, which read as
unreliable and blocked claims of absence even for complete lexical
coverage.

Validation: cargo test passes (375); plugin tests pass (37).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:40:16 -04:00
Joshua MoonandClaude Opus 5.5 3d2ca0fc94 feat(results): preview each hit with a snippet
Search, refs, and map hits carry a snippet: the first line inside the
span that mentions a query term or the hit's name, preferring code over
comments, trimmed to 120 characters. Lines output prints it on an
indented `  LINE: TEXT` line under the hit; JSON adds `snippet`. Pages
keep snippets while at least eight hits (or all remaining) fit, then
fall back to locators. Regex hits drop the name column the snippet
already shows.

Agents re-ran trufflepig queries as grep because hits had coordinates
only; grep answers location and text in one call.

Validation: cargo test passes (373); plugin and evaluation tests pass;
navigation replay completes every scenario in the same 2 calls with
~12% more output.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:35:23 -04:00
Joshua MoonandClaude Opus 5.5 6c43c11569 feat(workspace): search home member by default
An unselected query now searches only the home member, waiting up to
8 seconds for a freshly seeded home index to publish, and widens to all
members when home has no hits or no published index. Pages record the
outcome as `scope` and lines output appends it to the coverage line.
Explicit in:/ws:home/ws:all selectors never widen.

Transcripts showed agents' first workspace searches mixing unrelated
repositories into results, and fresh worktrees answering from other
members while home was warming.

Validation: cargo test passes (370); evaluation tests pass (60);
navigation replay scenarios all complete.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:28:24 -04:00
Joshua MoonandClaude Opus 5.5 e289641c1a fix(cli): plain help and line-based regex hits
Print --help and --version as plain, unbudgeted text; help ends with a
query and navigation summary. At the default budget, help was a
truncated JSON stub, and agents that asked for it switched to rg.

Make re: line-based like grep: ^ and $ match at line boundaries, and a
line with several matches yields one hit instead of repeats that
crowded pages (and interleaved across workspace members).

Validation: cargo test passes (369).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 13:22:30 -04:00
Joshua MoonandCodex 3204105ddd feat(agent): integrate Codex and Claude skills
Add personal skill installation for Codex and Claude Code, with
harness-specific session attribution and Claude SessionStart exports.
Preserve existing Claude hooks and configure narrow runtime access.

Share disk-backed spool and audit settings, correct audit parsing and
ordering, and stop router-owned child daemons during service upgrades.
Update the search workflow and navigation replay for compact results,
verified source reads, explicit relationship limits, and fallbacks.

Validation: 21 integration tests and 60 evaluation tests pass. Verify
four navigation replay scenarios, native skill discovery and Claude
hook execution, and search-to-source reads through the installed wrapper.
Automatic model selection and Claude-sandbox execution remain untested;
small exact-symbol comparisons do not establish token savings.

Co-authored-by: Codex <[email protected]>
2026-09-16 15:19:12 -04:00
Joshua MoonandClaude Fable 5.1 1aecebc089 feat(workspace): let linked worktrees stand in for members
Resolve the home member by Git common directory so a worktree of a
member, inside or outside its checkout, is searched under the member's
name with its own index cache, labelled in coverage as member@worktree.
Seed that cache from the member's extraction facts and embeddings,
exit a per-root daemon when its root disappears, sweep caches for
vanished roots from the router (also `system prune`), skip a linked
worktree's `.git` file, and teach the steer hook the same mapping.
Bound the history heartbeat stop so daemon exit never hangs.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-16 11:10:02 -04:00
Joshua MoonandClaude Fable 5.1 e5237674ed feat(cli): add lines output format for agents
Add --format lines for search, refs, map, more, and show: one tab
line per hit, a one-line coverage summary, and a key: value footer,
budgeted on the rendered text so pages carry more hits. JSON stays
the default and errors remain JSON. The agent wrapper injects the
flag, parses both shapes for its audit, and ignores stale session
markers; the skill documents the format and budget paging.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-16 09:42:44 -04:00
Joshua MoonandClaude Fable 5.1 597768b041 feat(system): serve sandboxed clients via file spool
Muse's seccomp filter denies AF_UNIX connects and leaves only /tmp
writable, so the router now drains /tmp/trufflepig-<uid>/spool on its
idle tick and clients spool a request when the socket is unreachable
and the router heartbeat is fresh. The agent wrapper's audit fallback
moves to /tmp/trufflepig-<uid>/agent-audit, which trufflepig-audit
also reads.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-16 09:06:18 -04:00
Joshua MoonandClaude Fable 5.1 433c062892 fix(workspace): name sandbox limits in error
Muse's seccomp filter returns EPERM on every AF_UNIX connect and its
cache mounts are read-only, so neither the router nor --no-daemon can
serve a sandboxed call. Say so instead of pointing at the router.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-15 23:33:08 -04:00
Joshua MoonandClaude Fable 5.1 b43712a1ef feat(plugin): run system router as user service
Ship a systemd user unit for `trufflepig system-serve` with
restart-on-failure and KillMode=process, install it with
`install.sh --systemd`, and have the session-start hook start the
service before falling back to a detached spawn. Sandboxed harnesses
could not recover once the router died; the coordinator error now
names the router and how to bring it back.

Also lands the search-steering hook: a PreToolUse hook that blocks or
nudges Grep/Glob/shell grep inside indexed repositories until
trufflepig-agent has been used, wired into Kimi and Muse, with its
decisions counted in trufflepig-audit. install.sh refreshes the
managed Kimi hook block on upgrade.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-15 20:27:26 -04:00
Joshua MoonandKimi-K3 77b52feb09 docs: document system daemon
Cover the router's purpose, socket dir resolution, system verbs,
session-start strap, fallback chain, and routing exclusions in cli.md;
add the router contract to the index contract's daemon section.

Co-authored-by: Kimi-K3 <[email protected]>
2026-09-15 18:56:33 -04:00
Joshua MoonandClaude Fable 5.1 3af8f710b0 feat(plugins): add Kimi and Muse agent plugin
Ship plugins/trufflepig-agent: a shared code-search skill, the
trufflepig-agent wrapper that tags requests by harness and session and
writes per-call JSONL audit records, the trufflepig-audit summarizer
that flags struggling sessions, Kimi session hooks, a Muse plugin
manifest, and an installer.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-15 14:51:55 -04:00
Joshua MoonandClaude Fable 5.1 b8446caf31 feat(workspace): add default output budget setting
Read [output].budget from workspace configuration and apply it when a
request passes no explicit -b/--budget. The space workspace uses 1200
so first pages hold about ten files instead of two.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-15 14:51:55 -04:00