Add caller/reference analysis, C# imports, and de-noised search
Agent-workflow tooling and search quality improvements: - find_callers / find_references (by id or exact name), with caller attribution to the innermost enclosing symbol and import-graph narrowing - index file (single-file re-index) and workspace glob member resolution - search symbols: exact > prefix > fulltext > substring fallback, inline source and line numbers, --limit on all search paths - search text: reads live from disk (no stale FTS snapshot), collapses duplicate snippets, correct multi-word phrase matching - find dead-symbols: excludes framework-wired entry points (pytest test_*, #[test] tests, FastAPI/Flask routes, Celery tasks, click/typer commands) - find references: filters the definition line and string/comment occurrences (incl. Rust raw strings) - C# import edges via new extract_csharp (using directives) - richer MCP tool descriptions; brace-in-string tokenizer fix Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
1 parent
a416a9fe52
commit
f1531a64d4
12 files changed
+1317
-214
No files matched your search
@@ -6,8 +6,8 @@ coding agent retrieve **symbols and functions** instead of whole files — savin
|
||||
context tokens.
|
||||
|
||||
Language coverage (via ctags): Rust, Java, Python, C/C++, TypeScript,
|
||||
JavaScript, HTML. HTML import edges are extracted directly (not via ctags), and
|
||||
`<link>`/`<script>`/`<img>` references to local `.css`/`.js`/image files are
|
||||
JavaScript, HTML, C#. HTML import edges are extracted directly (not via ctags),
|
||||
and `<link>`/`<script>`/`<img>` references to local `.css`/`.js`/image files are
|
||||
tracked as imports. CSS files are discovered and import-tracked, but ctags has
|
||||
no CSS parser — `search symbols` returns nothing for `.css` files.
|
||||
|
||||
@@ -41,15 +41,19 @@ cargo build --release
|
||||
|---|---|
|
||||
| `index create <path>` | Index a project from scratch |
|
||||
| `index update <path>` | Re-index only changed files (content-hash diff) |
|
||||
| `search symbols <query>` | Symbol search by name (exact > prefix > full-text) |
|
||||
| `search text <query>` | Full-text search over file contents |
|
||||
| `inspect symbol <id>` | Source of a single symbol |
|
||||
| `index file <file>` | Re-index a single file |
|
||||
| `search symbols <query> [--include-source] [--limit n]` | Symbol search (exact > prefix > full-text > substring) |
|
||||
| `search text <query> [--limit n]` | Full-text search with line numbers, read live from disk |
|
||||
| `inspect symbol <id\|name>` | Source of a single symbol (id or exact name) |
|
||||
| `inspect outline <file>` | Symbol outline of a file |
|
||||
| `inspect file <file> [--lines a-b]` | Read a file or line range |
|
||||
| `inspect tree [--depth n]` | Indexed file tree |
|
||||
| `inspect blast-radius <file> [--depth n]` | Files transitively affected by a change |
|
||||
| `find importers <file>` | Who imports this file |
|
||||
| `find dead` | Files nobody imports (entry points excluded) |
|
||||
| `find dead-symbols` | Symbols never referenced (framework-wired entry points excluded) |
|
||||
| `find callers <id\|name>` | Symbols that call the given symbol |
|
||||
| `find references <identifier>` | Code lines referencing an identifier (definitions/strings/comments excluded) |
|
||||
| `inspect cycles` | Circular imports |
|
||||
| `serve` | MCP server over stdio |
|
||||
| `stats` | Index statistics |
|
||||
@@ -76,6 +80,10 @@ rustytokens --project ./myproject find importers src/auth.rs
|
||||
rustytokens --project ./myproject find dead
|
||||
rustytokens --project ./myproject inspect cycles
|
||||
|
||||
# Who calls `authenticate` (id from `search symbols`) and where is it referenced?
|
||||
rustytokens --project ./myproject find callers 42
|
||||
rustytokens --project ./myproject find references authenticate
|
||||
|
||||
# Human-readable output
|
||||
rustytokens --project ./myproject --format table search symbols authenticate
|
||||
```
|
||||
@@ -96,10 +104,11 @@ Register the binary as a stdio MCP server (e.g. in Claude Code / Finir
|
||||
}
|
||||
```
|
||||
|
||||
Exposed tools: `index_create`, `index_update`, `search_symbols`,
|
||||
`search_text`, `inspect_symbol`, `inspect_outline`, `inspect_file`,
|
||||
`inspect_tree`, `find_importers`, `find_dead`, `inspect_cycles`,
|
||||
`inspect_blast_radius`, `stats`.
|
||||
Exposed tools: `index_create`, `index_update`, `index_file`,
|
||||
`search_symbols` (with `include_source` and `limit`), `search_text` (with
|
||||
`limit`), `inspect_symbol`, `inspect_outline`, `inspect_file`, `inspect_tree`,
|
||||
`find_importers`, `find_dead`, `find_dead_symbols`, `find_callers`,
|
||||
`find_references`, `inspect_cycles`, `inspect_blast_radius`, `stats`.
|
||||
|
||||
## Why it saves tokens
|
||||
|
||||
@@ -141,12 +150,10 @@ Notes:
|
||||
## Scope and known gaps
|
||||
|
||||
`rustytokens` covers the same *core* retrieval surface for Java/Python/Rust/
|
||||
C/C++/HTML as `toktoken`, but intentionally implements a smaller tool set: 13
|
||||
C/C++/HTML as `toktoken`, but intentionally implements a smaller tool set: 17
|
||||
tools vs `toktoken`'s 27. The following `toktoken` tools have **no equivalent
|
||||
here yet** (deliberately, not an oversight):
|
||||
|
||||
- `index_file` (reindex a single file)
|
||||
- `find_callers`, `find_references` (cross-reference search)
|
||||
- `inspect_bundle`, `inspect_dependencies`, `inspect_hierarchy` (graph views)
|
||||
- `search_similar`, `search_cooccurrence` (similarity search)
|
||||
- GitHub indexing and centrality ranking
|
||||
@@ -160,6 +167,16 @@ Known limits that are by design, not bugs:
|
||||
parser — `search symbols` returns nothing for `.css`.
|
||||
- `find dead` over-counts for C/C++ header-only entry points (documented
|
||||
conservatism); treat results as candidates, not verdicts.
|
||||
- `find callers` / `find references` are word-boundary heuristics: they match a
|
||||
symbol's name in text, so same-name symbols in unrelated files can produce
|
||||
false positives (`find callers` narrows this via the import graph).
|
||||
- `find dead-symbols` treats pytest `test_*` functions, `#[test]`-wired Rust
|
||||
tests and framework-decorated symbols (FastAPI/Flask routes, Celery tasks,
|
||||
click/typer commands, …) as live entry points. A pure name-scan cannot see
|
||||
these call sites, so this is a documented heuristic.
|
||||
- `find references` filters the definition line and occurrences inside strings
|
||||
and comments; multi-line docstrings/block comments are approximated
|
||||
line-by-line, so a symbol name inside a long string block may still appear.
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
Reference in new issue
Block a user