Launch HN: Moonglow (YC S24) – Serverless Jupyter Notebooks

On the other hand, if you are trying to run an entire notebook on a remote machine by starting a Jupyter server with Modal, then the workflow with Modal is not that different from other clouds (e.g. you can start an EC2 instance and run a Jupyter server there). For that, Moonglow still makes it easier by letting you stay in your IDE and avoid juggling Jupyter server URLs.

Also, you might need to use a specific cloud e.g. if you have cloud credits, sensitive data that needs to stay on that cloud or just expensive egress fees. One of Moonglow's strengths is that you can do your work in that cloud, rather than having to move stuff around.

For a lot of ML/AI workloads and tasks, Python is just a binding for underlying C/C++.

It's already a nightmare to try to reproduce any ML/AI paper, pip breaks 3 times, incompatible peer deps, some obscure library emits an obscure CLANG error that means I need to brew install some libwhatever, etc...

I don't think the WebAssembly toolchain is quite ready for plug and play "pip install" time yet. I hope it eventually will be though.

Hopefully someday you'll have 8 H100s on your Macbook, but I think we're still a long way away from that.

Yay I really can have serverless notebooks! Not just an easy to manage server environment but literally a static html file that can be passed around and runs the full notebook environment. It’s weird it was ever done any other way.

The big difference is that Google Colab runs in your web browser, whereas Moonglow lets you connect to compute in the VSCode/Cursor notebook interface. We've found a lot of people really like the code-completion in VSCode/Cursor and want to be able to access it while writing notebook code.

Colab only lets you connect to compute provided by Google. For instance, even Colab Pro doesn't offer H100s, whereas you can get that pretty easily on Runpod.

We don't yet transfer the python environment on the self-serve options, though for customers on AWS we'll help them create and maintain images with the packages they need.

I do have some ideas for making it easy to transfer environments over - it would probably involve letting people specify a requirements.txt and some apt dependencies and then automatically creating/deploying containers around that. Your idea of actually just detecting what's installed locally is pretty neat too, though.

[1] https://github.com/joouha/euporie

2. I think the serverless here is actually pretty literal - you don't have to think about or spin up a Jupyter server. We normally describe it as 'run local jupyter notebooks on your own cloud compute,' which I think might be a little more clear.

There's no SSH keys to store - we start a tunnel from the remote machine and connect to that.

One nice thing about our VSCode extension is that it's not just a remote kernel - our extension also lets you see what kernels you have and other details, so we'd need to write something like it for Zed. We probably wouldn't do this unless there's a lot of demand.

By the way, VSCode also supports the # %% repl spec and Moonglow does work with that (though we haven't optimized for it as much as we've optimized for the notebooks).

One thing I've found while working in the ML space is that it seems like ML researchers have to deal with a lot of systems cruft. I think that in the limit, ML researchers basically only care having about a few things set up well:

- secrets and environment management

- making sure their dependencies are installed

- efficient access to their data

- quick access to their code

- using expensive compute efficiently

But to get all this set up for their research they need to wade through a ton of documentation about git, bash, docker containers, mountpoints, availability zones, cluster management and other low-level systems topics.

I think there's space for something like Replit or Vercel for ML researchers, and Moonglow is a (very early!) attempt at creating something like it.

However, looking at its replacement here (https://docs.databricks.com/en/dev-tools/bundles/index.html) - I think we're trying to solve the same problems at different levels. My guess is Databricks is the right solution for big teams that need well-defined staging/prod/dev environment. We're targeting smaller teams that might be doing more of their own devops or are still at the 'using a bash script to run notebooks remotely' stage.

Moonglow abstracts over this, so you don't need to think about the server connection details at all. We're aiming for an experience where it feels like you've moved your notebook from local to cloud compute while staying in your editor.