Fix six open issues: stale imports, MCP validation, import-graph gaps, dup importers, I/O errors, scope docs

- pipeline: re-resolve imports for every file on update (not just changed),
  so add/remove/rename of a target fixes unchanged importers; propagate hash
  and read errors instead of treating unreadable files as empty
- discovery: report unreadable walk entries to stderr instead of dropping them
- imports: strip `pub` visibility so `pub use` re-exports resolve; share
  extension/module-entry probe lists (adds cc,cxx,hpp,hh,hxx,mts,cts,jsx,mjs,cjs)
- store: DISTINCT + ORDER in importers_of to dedup repeated edges
- mcp: validate required tool args (explicit isError), emit JSON-RPC -32700
  parse errors and -32600 invalid-request instead of silently skipping
- README: document the 13-tool surface vs toktoken's 27 and known gaps
- tests: regression coverage for update re-resolution, pub-use re-exports,
  importer dedup, and MCP error reporting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
endergebnisandCopilot committed 2026-09-04 18:49:06 +02:00
1 parent 6875998ab6
commit 4338e28ce0
7 files changed
+271 -49

No files matched your search

+23
View File
@@ -138,6 +138,29 @@ Notes:
- Hardware cycle counters were not available (`perf` tools missing on this
kernel), so cycle figures are estimates: CPU time × 3.2 GHz nominal.
## Scope and known gaps
`rustytokens` covers the same *core* retrieval surface for Java/Python/Rust/
C/C++/HTML as `toktoken`, but intentionally implements a smaller tool set: 13
tools vs `toktoken`'s 27. The following `toktoken` tools have **no equivalent
here yet** (deliberately, not an oversight):
- `index_file` (reindex a single file)
- `find_callers`, `find_references` (cross-reference search)
- `inspect_bundle`, `inspect_dependencies`, `inspect_hierarchy` (graph views)
- `search_similar`, `search_cooccurrence` (similarity search)
- GitHub indexing and centrality ranking
Add them only if the agent workflow actually needs them; they are not required
for the "roughly cover Java/Python/Rust/C/C++/HTML" goal.
Known limits that are by design, not bugs:
- `.css` files are discovered and import-tracked, but `ctags` has no CSS
parser — `search symbols` returns nothing for `.css`.
- `find dead` over-counts for C/C++ header-only entry points (documented
conservatism); treat results as candidates, not verdicts.
## Development
```bash