Why coding agents choose grep over LSP: a study of tool interfaces
Grep beats LSP? Why coding agents ignore your fancier tools
A small study compared grep's lexical search with LSP-backed semantic navigation in coding agents. Despite semantic tools offering more precise results, agents often stuck with grep, and forcing semantic-first paths sometimes reduced success. The key insight: a tool's LLM-friendliness depends on returning enough context in a usable shape, not just precision. Task type and codebase noise also matter—semantic navigation helped on noisy repos but added overhead on clean ones. The findings highlight that agent capability is a product of model and harness, and that preserving native tool loops, as AgentConnect does via ACP, may be crucial.
A tool is not friendly to a model merely because its results are precise. It must return enough context for the next step and present that context in an interface and output shape the model can use directly.
- tonyarkles
There’s been an interesting co-evolution that I’ve been experiencing with Claude Code. I’ll ask it to do a task, I’ll watch what it’s doing (often lots of find and grep and ripgrep) and then after the task is complete I’ll ask it if there are any tools that would’ve made the job easier. This has led to tools like fzf and others (notmuch for indexing email, for example). I’ve then taken those tools and figured out how to work them into my own workflow, both CLI and Emacs.
We’ve also collaborated on some Python tooling that takes a rather slow data format that I often have to process and analyze, indexed the whole corpus, and for analysis I can do (or Claude Code can) a single-pass conversion to Parquet which is then queryable with DuckDB. That tool has dramatically improved my turnaround time on one-off analysis tasks and as a Python CLI tool using Typer, the interface is also nicely discoverable for LLM harnesses to work with.
- nomilk
Tangental, but my LSP config breaks every few months. I don't bother to fix it anymore, I open an LLM in ~/dotfiles and complain until it works again, usually in a few minutes.
What's interesting is observing how much work this takes. (it gives me much more empathy toward my past self; how was a clumsy human supposed to know and reason about these things!?, especially when I hadn't touched the configs since a few months prior and had forgotten them almost entirely).
Most often there's 10-20 very small programs all working together to give the desired experience. The amount of minutes and tokens required to solve these seemingly simple problems like "My LSP isn't working" is sometimes much more than expected.
- the_duke
I've had great success with a tool I wrote that can print sparse ASTs for code.
https://github.com/theduke/smartedit
With the skill installed GPT 5.6 usually automatically uses it, and it reduces code exploration time and token usage significantly, for example by just printing the types and functions in a file without bodies, and only expanding when needed.
(note: it also has editing functionality, which doesn't work so well, since the models are heavily tilted towards common editing tools in post training)
- x-complexity
Setting up an LSP relies on 2 requirements to properly work:
(a) The LSP's tools being competently built & consistent, and
(b) the LLM using it having been properly trained to use LSPs in general.
Using a native tool like grep has the same 2 assumptions, but
(a) is satisfied due to ossification of grep's core features (a good thing), and
(b) is extra-satisfied because the LLM can be trained to properly use grep specifically, and not "20th variation of grep wrapped behind an LSP, but just different enough to throw curveballs".
On top of that, grep is almost always present in default Linux environments, so its presence is assumed & can be relied upon when needed.
- brunoborges
What has worked well for me is to have a SKILL that instructs to use the LSP more often than not.
LSP works best when using dependencies that are already compiled locally, but if all source is available, yeah... I still don't have a solid answer on which one is best.
But again, for already compiled dependencies (think Java bytecode), without LSP configs, the agent is likely going to attempt to extract binaries from JAR files, use grep and javap, and potentially attempt to decompile the .class files.