Show HN: Beating Pokemon Red with RL and <10M Parameters (opens in new tab)

Really cool work. It seems like some critical areas (team rocket, safari zone) rely on encoding game knowledge into the reward function somehow, which "smuggles in" external intelligence about the game. A lot of these are related to planning, which makes me wonder whether you could "bolt on" an LLM to do things like steer the RL agent, dynamically choose what to reward, or even do some of the planning itself. Do you think there's any low-hanging fruit on this front?

Note: What makes this interesting is that this is a pre-LLM project which shows that in some projects you don't need an "LLM" for this. All you need is just a plain old reinforcement learning algorithm and a deep neural network which is perfect for this.

This is what I want to see more of and goes against the hype of LLMs. What a great RL project.

Meanwhile, "Claude" is still stuck somewhere in the game. Imagine the costs of running that vs this project.

Wow nice work. 10M is a tiny model and I suspect this might be the future for specialised work. I can also imagine the progress towards AGI/ASI to have smaller models used as submodules.

brains basically have “modules” like this as well - neuronal columns that handle specialised tasks. For example when you’re driving on the road, the understanding whether the distance between you and the vehicle in front is increasing or decreasing is a finely tuned function of a specialised part of the brain.

Please stream the gameplay to twitch so people can compare.

Incredible work. I am just learning about PyBoy from your project, and it made me think of many fun ways to use that library to play Pokemon autonomously.

What an awesome project! I'm curious - I would have thought that rewarding unique coordinates would be enough to get the agent to (eventually) explore all areas, including the key ones. What did the agents end up doing before key areas got an extra reward?

(and how on earth did you port Pokémon red to a RL environment? O.o)

Can't Pokemon be beaten by almost random play?

Are there any uses for AI yet that aren't either:

1. Doing things humans do for fun. 2. Doing things that AI is horribly terrible at.

?

Awesome! Why do you think the reward for reading signs helped? I'm assuming the model doesn't gain the ability to read and understand english just from RL, so what purpose does it serve other than to maybe waste ticks on signs that ultimately don't need to be read?

Very nice! Nice to see demonstrations of reinforcement learning being used to solve non-trivial tasks.

This is very cool, congrats!

I wonder, does anyone have a sense of the approximate raw number of button presses required to beat the game? Mostly curious to see how that compares to the parameter count.

Heads up, clicking "Next Page" just takes you to an empty screen, you have to use the navigation links on the left if you want to get read past the first screen.

Ah, very neat.

Maybe some day the “rival” character in Pokemon can be played by a RL system, haha. That way you can have a “real player (simulated)” for your rival.

Really missing the arxiv link. The whole page reads like the arxiv link should be in the next paragraph, but it never appeared.

This is a first-in-world, isn't it?

Considering how many things are less complicated than Pokemon, this is very cool

> Pokémon Red takes 25 hours on average for a new player to complete.

Seriously? I've never really played video games, but i remember spending so much time on pokemon red when i was young. Not sure if i ever really finished more than once. But i'm pretty sure i must have played for more than 50h or so before even close to finish. My memory might trick me though.

Not sure which pokemon version it was, but i got so hooked trying to get this "secret" pokemon which was just a bunch of pixels. Some kind of bug (of the game, not the type of pokemon). You had to do specific things in a park and other things and then surf up and down x-times on the right shore of an island... or something like that. I had no idea how it worked and got so hooked, i must have spent most of my playing time on things like that.

Oh boy, memories...

Could you have used the decompilations of pokemon on github? https://github.com/pret/pokered