>Isn't speed limited by the amount of light that can be received in a small lens like on a phone?

It's also limited by sensor noise and efficiency.

Presumably, we'll push all three of these limits over the next few years.

The trick is to add the signal of many pixels together, forming a larger effective sample. The output video is very low-resolution and has a low dynamic range after noise removal, but can get hundreds of FPS in decent lighting.