Kradle: Eval AI with Simulations (opens in new tab)

Hi HN -

Minecraft has a DSL that lets you manipulate the world. We've piggy-backed on that, along with a K8s infra to run N worlds in parallel, to let you create simulations of arbitrary complexity.

We think simulations are the best way to test frontier AIs due to their degrees of freedom and expressivity.

AMA!

Interesting work! This can be a great way to have ML models live in a sandbox and prove out some of the safety concerns Anthropic et al speak about.

how do you see the world-simulation approach overlapping with the sort of custom-RL environments (e.g. mock Salesforce apps) that people are building out for frontier labs?

similarly, do you see it as a general test of intelligence? more for robotics?

very cool. how are ya'll thinking about genie 3?