Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep (opens in new tab)

What I have personally observed with such tools is that they make the AI's dumb, similar to how it makes coders dumb when relying more on AI tools.

These agentic AI's are already smart enough to figure out a highly optimized path to code exploration or search. But, with these tools, they just go very aggressive, partly because the search results from these tools almost in 100% of the cases do not furnish full details, but, just the pointers.

To confirm this behaviour, I did a small test run. This is in no way conclusive, but, the results do align with what I been observing:

---

Task: trace full ingestion and search paths in some okayish complex project. Harness is Pi.

1. With "codebase-memory-mcp": 85k/4.4k (input/output tokens).

2. With my own regular setup: 67k/3.2k.

3. Without any of these: 80k/3.2k.

As we see, such a tool made it worse (not by much, but, still). The outputs were same in quality and informational content.

---

Now, what my "regular setup" mentioned above is?:

Just one line in AGENTS.md and CLAUDE.md: "Start by reading PROJECT.md" .

And PROJECT.md contains just following: 2-3 line description of the project, all relevant files and their one-line description, any nuiances, and finally, ends with this line:

    ## To LLM
    Update this file if the changes you have done are worth updating here. The intent of this file is to give you a rough idea of the project, from where you can explore further, if needed.

Interesting. I too have been working in this space, though I took a different approach. Rather than building an index, I worked on making a "smarter grep" by offering search over codebases (and any text content really) with ranking and some structural awareness of the code. Most of my time was spend dealing with performance, and as a result it runs extremely quickly.

I will have to add this as a comparison to https://github.com/boyter/cs and see what my LLMs prefer for the sort of questions I ask. It too ships with MCP, but does NOT build an index for its search. I am very curious to see how it would rank seeing as it does not do basic BM25 but a code semantic variant of it.

This seems to work better for the "how does auth work" style of queries, while cs does "authenticate --only-declarations" and then weighs results based on content of the files, IE where matches are, in code, comments and the overall complexity of the file.

Have starred and will be watching.

I did some evals with pi and GPT 5.5. I tested RTK on / headroom on / both on / both off (all with the standard pi system instructions and no AGENTS.md).

I forget the exact tests I used (a couple of the standard agent evals that people use, one python and one typescript because those are what I use).

I don't claim it was an exhaustive test, or even a good one. It's possible I could have spent a day or so tuning my AGENTS.md and the pi system prompt/tool instructions and gotten better results, because if there's one thing running evals taught me it's that subtle differences there can change the results a lot.

However, I got clearly better results with both off, enough to convince me to stop the tests immediately after 3 rounds.

The problem was that while context use did go down (sometimes), the number of turns to complete went up so the overall cost of the conversation was higher.

It's made me very aware of one thing: so many people are sharing these kind of tools, but either with zero evals (or suspiciously hard to reproduce), or in the case of this one, extensive benchmarks testing the wrong thing.

I'm sure this tool does use fewer tokens than grep, and the benchmarks prove it, but that's not what matters here. What matters is, does an agent using it get the same quality of work done more quickly and for lower cost?

I'd be interested in seeing actual agent benchmarks (eg CC or Copilot CLI with grep removed and this tool instead).

For example, I have explored RTK and various LSP implementations and find that the models are so heavily RL'd with grep that they do not trust results in other forms and will continually retry or reread, and all token savings are lost because the model does not trust the results of the other tools.

Seems like a cool idea so I decided to play with it a bit. The test I ran was in the browsercode (https://github.com/browser-use/browsercode) repo with the following prompt:

"Answer this question by only using the `semble` CLI (docs below):

> What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work

---

[the AGENTS.md snippet provided from https://github.com/MinishLab/semble#bash-integration]"

And the equivalent for the non-Semble test:

"Answer this question by only using the `rg` and `fd` CLIs:

> What tools does Browsercode provide to the agent other than the base OpenCode tools? Provide the exact schema for tool input and tool output and briefly summarize what they do and how they work"

In both cases, I used Pi with gpt-5.4 medium and a very minimal setup otherwise. (And yes, I did verify that either instance only used rg & fd, or only used semble.)

Without Semble, it used 10.9% of the model context and used $0.144 of API credits (or, at least, that's what Pi reported - I used this with a Codex sub so cannot be sure). With Semble, it used 9.8% of the model context and $0.172 of API credits. The resulting responses were also about the same. Very close!

I tried one more test in the OpenCode repo. The question was > Trace the path from 1) the OPENCODE_EXPERIMENTAL_EXA env var being set to to 1 to 2) the resulting effects in the system prompt or tool provided to the OpenCode agent.

And I included the same instructions/docs as above. The non-Semble version was a bit more detailed -- it went into whether the tool call path invoked Exa based on whether Exa or Parallel was enabled for the web search provider -- but w.r.t. actually answering the question, both versions were accurate. The Semble version used 14.7% context / $0.282 API cost, while the non-Semble version used 19.0% / $0.352. Clearly a win for Semble for context efficiency, but note that the non-Semble version finished about twice as fast as the Semble version.

Of course this is just me messing around. ymmv.

I know this tool was meant for AI, but I am more interested in using it myself when exploring new code bases or even my own, when I want to refactor something and want an overview of where to change stuff. LSPs do that, but this tool sounds like it can go one step further.

Related read (not from me): https://entire.io/blog/improving-agentic-search-in-coding-ag...

> The clearest result was that faster search alone only modestly helps, while better-ranked results improve first-query retrieval and help agents find the right code sooner.

Their tool "pgr" is a research preview only, so it'd be interesting to see semble vs pgr.

I'm also collecting other tools that are similar, most notably is probably Morph's WarpGrep (has a free tier too). Apart from that, there is codemogger (https://github.com/glommer/codemogger), cs (the author also commented in this HN post).

In the similar area, but not fully related, the author of fff is also pretty involved in any thread that goes into that direction (see e.g. https://x.com/neogoose_btw/status/2052161471296225710). Similar to colGREP is also mgrep (by mixedbread) and osgrep (but they seem to predate colGREP). I also found codedb on X (https://codegraff.com/blog/codedb-code-intelligence), the post reads well, but haven't tried.

Nice, this sounds great. I want to mention a related issue here, which is that on small codebases, Claude spends a lot of time looking for stuff when it could have just dumped the whole codebase into the context in one go and used very little tokens.

I found a nice workaround which is that you can just dump the whole directory into context, as a startup hook. So then Claude skips the "fumble around blindly in the dark" portion of every task. (I've also seen a great project that worked on bigger repos where it'll give the model an outline with stubs, though I forget what it was called.)

Semantic code search seems like a useful tool for a human too. Not just for agents.

Better than grep obviously, but how does this compare to existing LSPs?

LSPs are already miles better than grep-like tools. This was true for humans as much as it is for LLMs. It's a shame that Claude still treats it as a second-class citizen (both in the app itself and in the training). A simple "remember to use LSP instead of grep" is usually enough to get it on the right track.

I also like the index feature form https://maki.sh Source code has a lot of structure, using a real parser instead of grepping and reading files can potentially save a lot of tokens

So about a year ago I wrote my own attempt at something like this using vector indexing and BM25 (the latest version uses CocoIndex, I had a custom coded solution using ChromaDB before). I wrote a comprehensive enough test set that showed performance increases on the quality of search results and reduction in token usage versus grep and rg. I haven't had time to really polish it but it worked well enough, particularly for one project where I have around 250k documentation files and docs out number code files 1000 to 1 (about 50% reduction in tokens and 30% increase in successful searches). Yesterday for grins I tried this project and was fairly disappointed to see it blow away my kludged solution particularly given that it doesn't have a lengthy indexing process. I haven't tested it on the 250k doc project yet, but in another project that I have a test suite for semantic search on it outperformed my solution by about 20% even on documentation in terms of successful search results (which I didn't expect given that it seems to only be tuned for code). I haven't gone through the code to see what its doing differently than what I tried, but what ever its doing it seems to have potential.

Related: https://github.com/jahala/tilth Edit: recent HN discussion: https://news.ycombinator.com/item?id=46952321

How does this compare with colgrep?

https://github.com/lightonai/next-plaid/tree/main/colgrep

Shouldn’t it be a part of the harness at least for local codebase? I wonder how many harnesses are doing that already.

Would be nice to see the actual % token/time saving across end-to-end coding sessions over these time periods https://github.com/MinishLab/semble#savings

My observation is that greps and the processing of grep outputs account for only a small portion of overall consumption; I haven't measured this scientifically though.

The bigger problem with solutions like these is that most AI already know how to use grep and search really well because of their training. Any such new tool that you handle to the AI, takes away from the cognitive capability of the AI. Humans would normally 'learn' how to operate tools like this - but the learning in LLM's is frozen and they already with a very strong depth in existing tools like grep.

For example, an AI would already use linux commands like tree to traverse the code base. And again it already has good training in this.

The other problem is that it is easy to cook up examples which demonstrate the efficacy of tools like these - but actually proving that the cognitive deficit that such tools result it, is surmounted by their efficacy in long horizon runs. My first contact instinct is that this will result in a net negative 'deployable intelligence' over long horizon runs - make the agent perform worse than using existing tools.

Proving the opposite is a non-trivial problem - but maybe it might be something you want to take up.

Metric that measures the quality beyond simple tokens count: correction loop frequency.

When grep does not find a file of interest, the agent does not fail; it will continue working on an incomplete context. For a monolingual code base, the miss rate is okay. In case of polylingual code (Python backend code and TypeScript frontend code), the problems emerge when it comes to querying for cross-file dependencies. Grep will return a route from the backend API. However, there is an interface in TypeScript that needs to be matched. Agent generates a response that does not fit the type. Correction cycle is one; two if the type conflict is ambiguous.

Combining grep with the understanding of semantic relations between files is a solution. Number of tokens saved is real but underestimates the actual benefit since fewer correction cycles are more valuable than tokens themselves.

Cool project. I built a custome IDE and coding agent harness and will integrate this into it. If you’re interested in a collaboration, I’d be happy to share revenue to sponsor your open source repo.

https://calendly.com/ryanwmartin/open-office-hours

Interesting for sure, but I think I need to be convinced this is necessary. How many tokens does Grep use on the average response anyways? Does this reduce input token usage from 100k to 2k per query or from 1k to 20? How does this effect output code quality?

> uses 98% fewer tokens than grep

So are we supposed to believe that grep is so wasteful that models are reading 98% useless garbage every time they call it? Either this claim is not representative, or you're missing something else when you throw away the vast majority of context for the model.

Feedback:

codex-cli hangs when calling this through the MCP. The semble process even sticks around as a zombie, forever stalled out. No idea why, logs have nothing.

When called through a skill via CLI style calling, GPT 5.5 loves to give a ton of search terms like it is used to doing with ripgrep. Not sure how effective this is, the short docs in the github and the instructions the agent has isn't clear on what is optimal.

Lastly, I got some errors with external connections to github when I was installing it for bash use. Maybe its related to the hanging? No idea.

edit: My agent also loves to follow-on with ripgrep, which seems redundant. Acts like it has trust issues. I think a more extensive agent skill description could guide the agent into proper use.

Would this replace something like codebase-memory-mcp[1] or improve when both is being used?

[1] - https://github.com/DeusData/codebase-memory-mcp

Is the benchmark measuring one-shot retrieval accuracy, or Coding agent response accuracy?

Does this work well for non-coding documents as well? Say api docs or AI memory files?

How does it compare to context-mode or serina that are both well established now?

This looks great! I built a tool in the same space- and I found that the biggest challenge was often to get the agent to prefer to use the tool over bash tools. What’s your experience with that?

fantastic token savings and performance... but unlike grep it's probabilistic search on search terms.

is that an issue? the tiny model might not surface something important

The instructions on how to install and use it could use some work. I did eventually install it. Will try it later and report back.

"The problem is that there are several bottlenecks internally," which include the requirements, specs, and testing. Another one not mentioned in the article.

Before you had faster implementation times, something would take six weeks to implement. Feedback from the client about how far off target you were came through in the same amount of time: a help desk ticket, a post-call check-in, a quarter end review. The price you paid for being off target was proportional to how long it took to figure out.

Now, when you can ship features in an afternoon, the customer feedback loop remains the same speed. Surveys, help desk tickets, and churn analysis come back days, even weeks later, by which point you've shipped five new features going the same way.

You can fix the internal bottlenecks easily enough: write better specs, have faster test cycles, deploy continuously. The customer feedback loop bottleneck is built into the system. It won't get any faster just because implementation did.

Today most organizations are busy fixing the internal bottleneck, but not the external one.

The savings are calculated against the assumption that agents read the matched files in their entirety. In my experience, they are smart enough to use grep to bring up a few lines of surrounding context, then read in full only the files that look relevant.

“> Claude: Semble surfaced things grep missed — here are the additions to the earlier answer.”

Nice!

The one thing I'm a bit nervous about: security. Thoughts of supply-chain "what-ifs" gives me a bit of pause here. Would like to hear security-minded folks give their thoughts on this.

Have you considered solving a different problem? What are agents trying to achieve when searching a code base? Finding seams. How about indexing seams instead?

How does that compare to Cursors’s workspace indexing?

Also curious what the authors think about Claude team explicitly trying out indexing and deciding against it.

Do inefficient grep/sed behaviors have a secondary benefit in seeding the context with breadcrumbs of code from irrelevant matches?

Congratulations on the release!

Could you add fff to the benchmarks?

Exciting. I've been playing with AI dev pipelines and the "give the agent the full codebase vs. let it search" trade-off is what I keep running into - both have pros and cons depending on the task. This looks like the latter pushed harder than I've seen before. Looking forward to trying it

Does this support any language or is it limited to a specific set of languages?

These almost always come with the cost of worse adherence

Interesting approach, gonna leave a gh star clap

grep doesn't need tokens, so what is 98% fewer than zero?

I feel this should be a mode on ripgrep. a disk search-lib in python? really?

Burntsushi (author of ripgrep), please chime in!

thanks for sharing!

amazing jobs

Nice!

Very useful thanks for sharing!

very curious to give it a spin but why write a cli in python? would surely be faster and more portable with go or rust?