FWIW, my quick impression is that takes reasonable concepts and tries to formalize them into a framework; I can see potential benefits, I've certainly asked in a claude code session for it to have a look at pipeline so and so and figure out the issue, but I'm not really convinced by this at first glance either. Both setup-cost and token cost seem like downsides.
I'd run the following 5-10 times with one model, then again with a 2nd model.
"Verify the correctness and completeness of all security configs/rules in SETUP.md. Consider if anything is missing, and if anything is not needed. Do not modify any files; only write potential findings to report.txt"
"Verify all findings and claims in report.txt."
Replace "SETUP.md" with whatever you're working on.
It's both terrifying and incredible watching what the models get correct and what they get completely wrong.
However, after enough runs they tend to settle on a state they claim does not need any more edits. And that result is generally useful with much fewer errors/hallucinations compared to a single run.
It's called review-anvil and does find significant amount of problems that might pop up:
Or does "verify all claims in report" counteract that?
E.g. some findings first classified as moderate priority often get reclassified as low priority even if the finding itself is correct.
The exact phrasing doesn't seem to matter as much as keeping the prompts short, simple and to the point.
However some models seem to do a bit better when adding ", if any" to prompts such as "List potential improvements".
Why not just look through the actual Claude code codebase and use your own AI to deconstruct it
