Multi-Agent Coordination in Factorio: FLE v0.2.0 (opens in new tab)

(jackhopkins.github.io)

13 pointsnoddybear10mo ago5 comments

5 comments

Hey everyone,

It's Mart, Neel and Jack from the Factorio Learning Environment team.

Since our initial release, we have been working hard to expand the environment to support multi-agent scenarios, reasoning models and MCP for human-in-the-loop evals.

We have also spent time experimenting with different ways to elicit more performance out of agents in the game, namely tools for vision and reflection.

Today, we are proud to release v0.2.0, which includes several exciting new features and improvements.

Thanks for checking this out.

goa10mo ago

I have a few qualms with this app:

1. For a competent ML researcher, this is trivial to build using VQA + MCTS + standard message passing.

2. MARL techniques with attention-based communication solved this years ago

3. It does not seem very "viral" or income-generating. I know this is premature at this point, but without charging users for the service, is it reasonable to expect to make money off of this?

noddybearOP10mo ago

1. These are additions to our existing Factorio Learning Environment, which is an extensive agent environment for evaluating pre-trained LLM agents in an unbounded/open-ended setting in the game of Factorio. I don't agree that it is trivial, as there is significant infrastructure in place to support Factorio as an LLM eval.

2. Factorio is an unsolved game in multi-agent research.

3. This is a research environment. You can read our paper on Arxiv if you're interested! Nobody will make any money of this.

goa10mo ago

Ok you are right, my initial post was tongue in cheek and aimed to reference the dropbox hackernews meme.

1. Are current agents running on the same machine, and with the future A2A they can play across the internet?

2. How many agents can play/cooperate right now?

3. Can you also control enemies? Might be fun to have a generative adverserial setup?

georgeh4cks10mo ago

Very cool! Big Factorio fan so excited to check this out!

j / k navigate · click thread line to collapse

5 comments

noddybearOP10mo ago

Hey everyone,

It's Mart, Neel and Jack from the Factorio Learning Environment team.

Since our initial release, we have been working hard to expand the environment to support multi-agent scenarios, reasoning models and MCP for human-in-the-loop evals.

We have also spent time experimenting with different ways to elicit more performance out of agents in the game, namely tools for vision and reflection.

Today, we are proud to release v0.2.0, which includes several exciting new features and improvements.

Thanks for checking this out.

goa10mo ago

I have a few qualms with this app:

1. For a competent ML researcher, this is trivial to build using VQA + MCTS + standard message passing.

2. MARL techniques with attention-based communication solved this years ago

3. It does not seem very "viral" or income-generating. I know this is premature at this point, but without charging users for the service, is it reasonable to expect to make money off of this?

noddybearOP10mo ago

2. Factorio is an unsolved game in multi-agent research.

3. This is a research environment. You can read our paper on Arxiv if you're interested! Nobody will make any money of this.

goa10mo ago

Ok you are right, my initial post was tongue in cheek and aimed to reference the dropbox hackernews meme.

1. Are current agents running on the same machine, and with the future A2A they can play across the internet?

2. How many agents can play/cooperate right now?

3. Can you also control enemies? Might be fun to have a generative adverserial setup?

georgeh4cks10mo ago

Very cool! Big Factorio fan so excited to check this out!

j / k navigate · click thread line to collapse