PyTorch Landscape (opens in new tab)

Rather than forking PyTorch (which has issues like continually needing updates), could you create a set of linter rules instead?

It's useful to be able to run models/code on the CPU or split between CPU and GPU, especially for models that cannot fit into the GPU's VRAM, if you are running multiple models, or if you have training data that you need to move between CPU and GPU.

> One thing I wonder is why no one has made a fork of PyTorch yet that […]

Try and compile the stack from source and you'll find out why nobody is making forks with small divergences.

Seems like you could write a simple source code checker program to check all of that. Making an extra library just for some (user hostile) tweaks seems like overkill.

Hi!

I'm curious to hear about your work geometric optics with PyTorch. May I ask you to share some examples of something you are working on right now?

We do try to check the pulse of projects there with a checklist, but I guess it started in a more uncoordinated way.

If you file an issue here, I think it would work to update things:

https://github.com/pytorch-fdn/ecosystem

I'm a computational mathematician, and I've been a big proponent of JAX to all the research groups I've worked with. So far, I've converted all of them =). In particular for math, I've found JAX's design and feature set to be far more suitable. Patrick Kidger's suite of libraries also carries a lot of weight in that regard. Equinox is the center is basically everything I write in JAX.

But there are absolutely pain points. In particular, especially for weaker programmers (common in academia), it's quite easy to write bad JAX. The functional programming and stateless paradigms require a little more thinking ahead. This is particularly tough when you're doing research and you're, in real time, finding out what's ahead!

JAX is brilliant for research but the debugging story is still rough compared to PyTorch. eager mode + native Python exceptions win for most people.

> after it seemed for a while that the dust had settled on Tensorflow.

Did it ever? PyTorch always seemed more popular to me, unless you were around Google people, then obviously everything was Tensorflow. But like many Google projects, I never got the impression it was as ubiquitous as PyTorch out in the community outside of Google. Maybe around 2018-2019 it felt like Tensorflow would become more popular than PyTorch eventually, but it never seemed to actually happen.

why did tensorflow not gain traction? out of the loop here

The compiler based approach, with jax.cond, purity expectations, difficulty of making stateful abstractions, etc makes sense for large scale deployment, especially to purpose built accelerators like the systolic array ones (tpu, trainium) but not for being the primary framework used in the field. Imperative always wins. Side effects and state are necessary.