This is especially true when you use a lot of tooling. I love jupyter, but installing it in a venv means pulling a lot of deps which will affect a lot what I can install.

Fortunately the Python community is much more serious about making deps that work together than the JS community, and the fact it works at all given the cartesian products of all the python modules is kind of a miracle and a testament to that.

Unfortunately, that's a problem that is unlikely to be solved in the next decade, so we all live with it.

The reverse problem is true for JS, and I see many projects shipping very heavy frontend code because despite all the tree shaking, they embed 5 times the same module with different versions in their bundle. That's one of the reasons for the bloated page epidemic.

I guess it's a trade-off for all scripting languages: choosing between bloat or compat problem. Rust and Go don't care as much, and on top of that they can import code from 10 years ago and it sill works.

However, and while I do know how hard it is to ship python code to the end user (at least if you don't use a web app), I don't think the version problem is the reason. We have zipapp and they work fine.

No the main reason iscompiled extensions are very useful and popular, which means packaging is solving more than packaging python, but a ton of compiled languages at one. Take scipy: they have c, pascal and assembly in there.

This can and will be improved though. In fact, thanks to wheels and indygreg/python-build-standalone, I think we will see a solution to this in the coming years.

I'm even betting on astral to providing it.

The problem I see a lot of JS developers having when they start using Python is they try to do the "import the entire world" strategy of development that's common in JS, and there isn't good tooling for that because Python just doesn't have that culture. And that's because it's a bad idea--it's not a better idea in JS, it's just more part of the culture.

Pick one package source. Stick with it. And don't import every 0.0.x package from that package source either.

There are obviously reasons to use more than one package source, but those reasons are far rarer than a lot of inexperienced devs think they are. A major version number difference in one package isn't a good reason to complicate your build system unless there are features you genuinely need (not "would be nice to have", need).

And it doesn't provide any way to use a link to any other package repository if you want to stick to vanilla pyproject.toml + build (the official build tool). So if you want to use the CUDA or rocm version of torch, for example, you have to add a direct link to the package. That means that you'd have to hardlink to a platform specific version of the package. There's no way to just make a package look at a non pypi repository to get the version you want otherwise.

So say you want to add pytorch, with GPU acceleration if it's possible on a platform. You want to make it multiplatform to some extent. You can't add another index if you want to use vanilla build, as that's not allowed. You can add a direct link (that's allowed, just not an index) but that's going to be specific to a platform+python version. Pytorch doesn't even provide CUDA packages on pypi anymore (due to issues pypi), so you need to be able to use another index! You'd need to manually create requirements.txt for each platform, create a script that packages your app with the right requirement.txt, and then do it again whenever you update. Otherwise, I think the most recent advice I've seen was to just make... the user download the right version. Mhmmmm.

The other option is to use poetry or something like that, but I just want to use "python build . "...

Poetry at least helps with that for Python. It's all still a mess though.

But you can do that, obviously not with this syntax. It’s non standard but I have built programs that install all dependencies as a first step. It’s pretty trivial.