undefined | Better HN

0 pointsorf1y ago0 comments

So many misunderstandings here :/ I can’t muster the energy to correct them past these two obvious ones

> It works on any Python version that all of the dependencies work on

No, it doesn’t. It’s not a lockfile: it’s a snapshot of the dependencies you have installed.

The dependencies you have installed depend on the Python version and your OS. The obvious case would be requiring a Linux-only dependency on… Linux, or a package only required on Python <=3.10 while you’re on 3.11.

> PyPI doesn't allow you to replace the package for the same version

Yes and no. You can continue to upload new wheels (or a sdist) long after a package version is initially released.

0 comments

zahlman1y ago

>So many misunderstandings here :/

I've spent most of the last two years making myself an expert on the topic of Python packaging. You can see this through the rest of the thread.

>No, it doesn’t. It’s not a lockfile: it’s a snapshot of the dependencies you have installed.

Yes, it does. It's a snapshot of the dependencies that you have installed. For each of those dependencies, there is some set of Python versions it supports. Collectively, the packages will work on the intersection of those sets of Python versions. (Because, for those Python versions, it will be possible to obtain working copies of each dependency at the specified version number.)

Which is what I said.

> The dependencies you have installed depend on the Python version and your OS. The obvious case would be requiring a Linux-only dependency on… Linux, or a package only required on Python <=3.10 while you’re on 3.11.

A huge amount of packages are pure Python and work on a wide range of Python versions and have no OS dependency. In general, packages may have such restrictions, but do not necessarily. I know this because I've seen my own code working on a wide range of Python versions without making any particular effort to ensure that. It's policy for many popular packages to ensure they support all Python versions currently supported by the core Python dev team.

Looking beyond pure Python - if I depend on `numpy==2.2.1` (the most recent version at time of writing), that supports Python 3.10 through 3.13. As long as my other dependencies (and the code itself) don't impose further restrictions, the package will install on any of those Python versions. If you install my project on a different operating system, you may get a different wheel for version 2.2.1 of NumPy (the one that's appropriate for your system), but the code will still work. Because I tested it with version 2.2.1 of NumPy on my machine, and version 2.2.1 of Numpy on your machine (compiled for your machine) provides the same interface to my Python code, with the same semantics.

I'm not providing you with the wheel, so it doesn't matter that the wheel I install wouldn't work for you. I'm providing you(r copy of Pip) with the package name and version number; Pip takes care of the rest.

>You can continue to upload new wheels (or a sdist) long after a package version is initially released.

Sure, but that doesn't harm compatibility. In fact, I would be doing it specifically to improve compatibility. It wouldn't change what Pip chooses for your system, unless it's a better match for your system than previously available.

orfOP1y ago

Holy hell dude, you don’t need to write a novel for every reply. It’s not a lockfile because it’s a snapshot of what you have installed. End of.

It doesn’t handle environment markers nor is it reproducible. Given any non-trivial set of dependencies and/or more than 1 platform, it will lead to confusing issues.

Those confusing issues are the reason for lock files to exist, and the reason they are not just “the output of pip freeze”.

But you know this, given your two years of extensive expert study. Which I see very little evidence of.

zahlman1y ago

>It’s not a lockfile because it’s a snapshot of what you have installed.

I didn't say it was. I said that it solves the problems that many people mistakenly think they need a lockfile for.

(To be clear: did you notice that I am not the person who originally said "'pip freeze' generates a lockfile."?)

>It doesn’t handle environment markers nor is it reproducible.

You can write environment markers in it (of course you won't get them from `pip freeze`) and Pip will respect them. And there are plenty of cases where no environment markers are applicable anyway.

It's perfectly reproducible insofar as you get the exact specified version of every dependency, including transitive dependencies.

>Given any non-trivial set of dependencies and/or more than 1 platform, it will lead to confusing issues.

Given more than 1 platform, with differences that actually matter (i.e. not pure-Python dependencies), you cannot use a lockfile, unless you specify to build everything from source. Because otherwise a lockfile would specify wheels as exact files with their hashes that were pre-built for one platform and will not work on the others.

Anyway, feel free to show a minimal reproducible example of the confusion you have in mind.

1 more reply

j / k navigate · click thread line to collapse