I didn't dig into what the actual repository was doing, but personally, I took some inspiration from the idea after reading about it and realizing that I might have been underestimating the ability of LLMs. I put a bit more work into a performance harness I was using locally and just set some agents to brainstorming and they did seem to find some great stuff. So I don't really have a stance one way or another on this specific repo, but the general idea seems like a really good one.

Ever since AlphaEvolve - the idea that if you build a harness which can evaluate solutions and give LLMs a database where they can keep storing their work and then sample from it - they do find non-trivial solutions over time leaning from their own past ideas.

It is the ultimate manifestation of test-time scaling. I think karpathy just popularised it.

Karpathy embedded within an organization is way more impressive than him out on his own with hot takes and little projects. I hope he does great things for Anthropic.

I was trying to look options outside the box (everything is more context or RAG) and been using this approach for about a month with good results. https://github.com/VDP89/fscars

I was impressed that I was able to take the same basic idea and apply it to anything that a Claude could construct a metric for. It's nice being able to just run /autoresearch and speed up your test suites, and shave time off your builds etc.

It's a decent tool to have in the toolbox.

    > Am I the only one who wasn’t particularly impressed by AutoResearch?

isn't it just a nerfed AlphaEvolve? https://arxiv.org/abs/2506.13131

Inefficient variants with $100m+ worth of compute will still probably outperform the best team of researchers