Depending on the domain, the reality can be the reverse.
Multiprocessing in the web serving domain, as in "spawning separate processes", is actually simpler and less bug-prone, because there is considerably less resource sharing. The considerably higher difficulty of writing, testing and debugging parallel code is evident to anybody who's worked on it.
As for the overhead, this again depends on the domain. It's hard to quantify, but generalizing to "massive" is not accurate, especially for app servers with COW support.
The default for multiprocessing is still to fork (fortunately changing in 3.14), which means all of your parent process’ threaded code (incl. third party libraries) has to be fork-safe. There’s no static analysis checks for this.
This kind of easy to use but incredibly hard to use safely library has made python for long running production services incredibly painful in my experience.
[1] Some arguments to subprocess.popen look handy but actually cause python interpreter code to be executed after the fork and before the execve, which has caused production logging-related deadlocks for me. The original author was very bright but didn’t notice the footgun.
Also, I've found that ChatGPT/Claude3.5 are much, much smarter and better at Python than they are at C++ or Rust. I can usually get code that works basically the first or second time with Python, but very rarely can do that using those more performant languages. That's increasingly a huge concern for me as I use these AI tools to speed up my own development efforts very dramatically. Computers are so fast already anyway that the ceiling for optimization of network oriented software that can be done in a mostly async way in Python is already pretty compelling, so then it just comes back again to developer productivity, at least for my purposes.
It will be interesting to see how this goes over the next few years. My guess is that a lot of lessons were learned from the python 2 to 3 move. This plan seems pretty solid.
And of course there's a relatively easy fix for code that can't work without a GIL: just do what people are doing today and just don't fork any threads in python. It's kind of pointless in any case with the GIL in place so not a lot of code actually depends on threads in python.
Preventing the forking of threads in the presence of things still requiring the GIL sounds like a good plan. This is a bit of meta data that you could build into packages. This plan is actually proposing keeping track of what packages work without a GIL. So, that should keep people safe enough if dependency tools are updated to make use of this meta data and actively stop people from adding thread unsafe packages when threading is used.
So, I have good hopes that this is going to be a much smoother transition than python 2 to 3. The initial phase is probably going to flush out a lot of packages that need fixing. But once those fixes start coming in, it's probably going to be straightforward to move forward.
AMD EPYC 9754 with 128-cores/256-threads, and EPYC 9734 with 112-cores/224-threads. TomsHardware says they "will compete with Intel's 144-core Sierra Forest chips, which mark the debut of Intel's Efficiency cores (E-cores) in its Xeon data center lineup, and Ampre's 192-core AmpereOne processors".
What in 5 years? 10? 20? How long will "1 core should be enough for anyone using Python" stand?
A piece of code takes 6h to develop in C++, and 1h to run.
The same algorithm takes 3h to code in Python, but 6h to run.
If I could thread-spam that Python code on my 24 core machine, going Python would make sense. I've certainly been in such situations a few times.
Every DL library comes with its own C++ backend that does this for now, but it's annoyingly inflexible. And dealing with GIL is a nightmare if you're dealing with mixed Python code.
IDK what l should and shouldn't be written in, but there are a very large # of proud "pure Python" libraries on GitHub and HN.
The ecosystem seems to even prefer them.
Why shouldn't someone who prefers writing in python benefit from using multiple cores?
This just isn’t true.
This does not improve single threaded performance (it’s worse) and concurrent programming is already available.
This will make it less annoying to do concurrent processing.
It also makes everything slower (arguable where that ends up, currently significantly slower) overall.
This way over hyped.
At the end of the day this will be a change that (most likely) makes the existing workloads for everyone slightly slower and makes the lives of a few people a bit easier when they implement natively parallel processing like ML easier and better.
It’s an incremental win for the ML community, and a meaningless/slight loss for everyone else.
At the cost of a great. Deal. Of. Effort.
If you’re excited about it because of the hype and don’t really understand it, probably calm down.
Mostly likely, at the end of the day, it s a change that is totally meaningless to you, won’t really affect you other than making some libraries you use a bit faster, and others a bit slower.
Overall, your standard web application will run a bit slower as a result of it. You probably won’t notice.
Your data stack will run a bit faster. That’s nice.
That’s it.
Over hyped. 100%.
The rest of us can live with arcane threading bugs and yet another split ecosystem. As I understand it, if a single C-extension opts for the GIL, the GIL will be enabled.
Of course the invitation to experiment is meaningless. CPython is run by corporations, many excellent developers have left and people will not have any influence on the outcome.
But, for sure, nogil will be good for those workloads written in pure Python (though I've personally never been affected by that).
I use coroutines and multiprocessing all the time, and saturate every core and all the IO, as needed. I use numpy, pandas, xarray, pytorch, etc.
How did this terrible GIL overhead completely went unnoticed?
That means your code is using python as glue and you do most of your work completely outside of cPython. That's why you don't see the impact - those libraries drop GIL when you use them, so there's much less overhead.
I've never heard threading described as "simple", even less so as simpler than multiprocessing.
Threads means synchronization issues, shared memory, locking, and other complexities.
Everyone wants parallelism in Python. Removing the GIL isn't the only way to get it.
I'm saturating 192cpu / 1.5TBram machines with no headache and straightforward multiprocessing. I really don't see what multithreading will bring more.
What are these massive overheads / complexity / bugs you're talking about ?
[x] Async.
[x] Optional static typing.
[x] Threading.
[ ] JIT.
[ ] Efficient dependency management.This is not a requirement for a language to be statically typed. Static typing is about catching type errors before the code is run.
> Type hint a var as a string then set it to an int, that code still gonna try to execute.
But it will fail type checking, no?
So you can do things like “from typing import Optional” to bring Optional into scope, and then annotate a function with -> Optional[int] to indicate it returns None or an int.
Unlike a system using special comments for type hints, the interpreter will complain if you make a typo in the word Optional or don’t bring it into scope.
But the interpreter doesn’t do anything else; if you actually return a string from that annotated function it won’t complain.
You need an external third party tool like MyPy or Pyre to consume the hint information and produce warnings.
In practice it’s quite usable, so long as you have CI enforcing the type system. You can gradually add types to an existing code base, and IDEs can use the hint information to support code navigation and error highlighting.
Works pretty efficiently.
BTW, Typescript also does not enforce types at runtime. Heck, C++ does not enforce types at runtime either. It does not mean that their static typing systems don't help during at development time.
python -c "x: int = 'not_an_int'"
My opinion is that with PEP 695 landing in Python 3.12, the type system itself is starting to feel robust.
These days, the python ecosystem's key packages all tend to have extensive type hints.
The type checkers are of varying quality; my experience is that pyright is fast and correct, while mypy (not having the backing of a Microsoft) is slower and lags on features a little bit -- for instance, mypy still hasn't finalized support for PEP 695 syntax.
The other tools are trivially easy to set up and run (or let your IDE run for you.) As in, one command to install, one command to run. It's an elegant compromise that brings something that's sorely needed to Python, and users will spend more time loading the typing spec in their browser than they will installing the type checker.
what it has is "type hints" which is way to have richer integration with type checkers and your IDE, but will never offer more than that as is
For efficient dependency management, there is now rye and UV. So maybe you can check all those boxes?
So there's plenty of well-founded hope, but the boxes are still not checked.
[X] print requires parentheses[0] https://stackoverflow.com/questions/56262012/conda-install-t...
I regularly encounter python code which takes minutes to execute but runs in less than a second when replacing key parts with compiled code.
This is a big fundamental and (in many cases breaking) change, even if it's "optional".
There were a lot of smaller breaking changes over the years, especially 3.10 that probably should have been a 4.0.
I’m looking forward to seeing how people use a Python that can be meaningfully threaded. While It may take a bit to built momentum, I suspect that in a few years there’ll be obvious use cases that are widely deployed that no one today has even really considered.
There have been patches to remove the GIL going back to the 90s and Python 1.5 or thereabouts. But the performance impact has always been the show-stopper.
So the net is actually a small performance win but lesser than if there was no free threading. That said, many of the techniques he identified were immediately incorporated into CPython and so I would expect benchmarks to show some regression as compared with the single threaded interpreter of the previous revision.
Meanwhile what takes the crown? - Single threaded python.
(Well, ok Rust looks like it's taking first place where you really need the speed and it does help parallelism without requiring absolute purity)
Any python library that cares about performance is written in C/C++/Rust/Fortran and only provides a python interface.
ML will have 0 benefit from this.
Is there a cibuildwheel / CI check for free-threaded Python support?
Is there already a reason not to have Platform compatibility tags for free-threaded cpython support? https://packaging.python.org/en/latest/specifications/platfo...
Is there a hame - a hashtaggable name - for this feature to help devs find resources to help add support?
Can an LLM almost port in support for free-threading in Python, and how should we expect the tests to be insufficient?
"Porting Extension Modules to Support Free-Threading" https://py-free-threading.github.io/porting/
[1] "Python 3 "Wall of Shame" Becomes "Wall of Superpowers" Today" https://news.ycombinator.com/item?id=4907755
(Edit)
Compatibility status tracking: https://py-free-threading.github.io/tracking/
python-feedstock / recipe / meta.yml: https://github.com/conda-forge/python-feedstock/blob/master/...
pypy-meta-feedstock can be installed in the same env as python-feedstock; https://github.com/conda-forge/pypy-meta-feedstock/blob/main...
sudo dnf install python3.13-freethreading
sudo add-apt-repository ppa:deadsnakes
sudo apt-get update
sudo apt-get install python3.13-nogil
conda create -n nogil -c defaults -c ad-testing/label/py313_nogil python=3.13
mamba create -n nogil -c defaults -c ad-testing/label/py313_nogil python=3.13
I'd love to see a more fluid model between the two -- E.G. if I'm doing a "gather" on CPU-bound coroutines, I'm curious if there's something that can be smart enough to JIT between async and multithreaded implementations.
"Oh, the first few tasks were entirely CPU-bound? Cool, let's launch another thread. Oh, the first few threads were I/O-bound? Cool, let's use in-thread coroutines".
Probably not feasible for a myriad of reasons, but even a more fluid programming model could be really cool (similar interfaces with a quick swap between?).
If you're serving HTTP requests, for instance, simply serving each request on its own thread with its own event loop should be sufficient at scale. Multiple requests each with CPU-bound tasks will still saturate the CPUs.
Very little code teeters between CPU-bound and io-bound while also serving few enough requests that you have cores to spare to effectively parallelize all the CPU-bound work. If that's the case, why do you need the runtime to do this for you? A simple profile would show what's holding up the event loop.
But still, the runtime can't naively parallelize coroutines. Coroutines are expected not to be run in parallel and that code isn't expected to be thread safe. Instead of a gather on futures, your code would have been using a thread pool executor in the first place if you'd gone out of your way to ensure your CPU-bound code was thread safe: the benefits of async/await are mostly lost.
I also don't think an event loop can be shared between two running threads: if you were to parallelize coroutines, those coroutines' spawned coroutines could run in parallel. If you used an async library that isn't thread safe because it expects only one coroutine is executing at a time, you could run into serious bugs.
This is exactly where I'd like to see it.
I'd like to simultaneously:
1. Call out to external APIs and not run any overhead/complexity of creating/managing threads 2. Call out to a model on a CPU and not have it block the event loop (I want it to launch a new thread and have that be similar to me) 3. Call out to a model on a GPU, ditto
And use the observed resource CPU/GPU usage to scale up nicely with an external horizontal scaling system.
So it might be that the async API is a lot easier to use/ergonomic then threads. I'd be happy to handle thread-safety (say, annotating routines), but as you pointed out, there are underlying framework assumptions that make this complicated.
The solution we always used is to separate out the CPU-bound components from the IO-bound components, even onto different servers or sidecar processes (which, effectively, turn CPU-bound into IO-bound operations). But if they could co-exist happily, I'd be very excited. Especially if they could use a similar API as async does.
Maybe if you’ve got an embarrassingly parallel problem, and dozen(s) of cores to spare, you can match the performance of a single-threaded JIT/AOT compiled program.
It’s much worse except in everything but a threaded test
-Episode 2: Removing the GIL[1]
-Episode 12: A Legit Episode[2]
[1]https://www.youtube.com/watch?v=jHOtyx3PSJQ&list=PLShJCpYUN3...
[2]https://www.youtube.com/watch?v=IGYxMsHw9iw&list=PLShJCpYUN3...
What about simple operations like incrementing an integer? IIRC this is currently thread-safe because the GIL guarantees each bytecode instruction is executed atomically.
I guess the only things that are a single instruction are some modifications to mutable objects, and those are already heavyweight enough that it’s OK to add a per-object lock.
I've done quite a bit of stuff with Java and Kotlin in the past quarter century and it's interesting to see how much things have evolved. Early on there were a lot of people doing silly things with threads and overusing the, at the time, not so great language features for that. But a lot of that stuff replaced by better primitives and libraries.
If you look at Kotlin these days, there's very little of that silliness going on. It has no synchronized keyword. Or a volatile keyword, like Java has. But it does have co-routines and co-routine scopes. And some of those scopes may be backed by thread pools (or virtual thread pools on recent JVMs).
Now that python has async, it's probably a good idea to start thinking about some way to add structured concurrency similar to that on top of that. So, you have async stuff and some of that async stuff might happen on different threads. It's a good mental model for dealing with concurrency and parallelism. There's no need to repeat two decades of mistakes that happened in the Java world; you can fast forward to the good stuff without doing that.
Really excited about this.
With it, the single-threaded case is slower.
The link should have been to https://py-free-threading.github.io/tracking/