Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle
Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that.
I am surprised that gevent was dismissed. Ok, there is also eventlet, if someone doesn't like gevent. Monkey patching is scary? Is it really _that_ scary? Most sane and portable IO code probably runs on that today. Why? Because there is no need to create a parallel world of libraries. Write a test does it pass? Does it handle your use case? I'll take not knowing if my green threads switch tasks and add another green thread lock, instead of doubling my code size with yields, callbacks and futures.
Let's talk about Twisted (sorry couldn't resist WAT video reference). I remember for years searching for parallel libraries to parse exotic protocols. Regular Python library is there, but no, can't use that, sorry. Gotta go find or write one that returns Deferreds. You add a single Twisted module in your code, good luck! -- it ripples all the way to the top through your API and you are doomed being locked into the Twisted world forever.
When gevent and eventlet came around it was like a breath of fresh air. This is what sane concurrent IO looks like in Python:
http://eventlet.net/doc/examples.html
My fear is that many will just say fuck it, I'll just use Go/Rust/Erlang for IO bound concurrent problems.
It is nice having a benevolent dictator, except when he goes a little crazy, then dictatorship doesn't sounds so much fun anymore.
In the case of async I'm glad to see a from scratch implementation for the standard library. It's a weird area that necessitates some constructions that there is no really standard Python style for. You only need to look at Twisted and e.g. its method chaining to realize this stuff would need a thorough sanity rework before it ever became standard anyway.
Also, most other implementations take the approach of building their own little world. This is definitely true of Twisted. You write code for Twisted, not for Python. Gevent at least doesn't suffer from this.
If it were integrated with Python, there would be no monkeypatching, no special magic, it would be just how things work. That said, I'm not at all surprised that Guido doesn't favor a coroutine-based solution; his opposition to general coroutines is as famous as his opposition to anonymous functions. (to clarify: I don't think the @coroutine decorator creates "real" coroutines, any more than generators were already coroutines)
Do you think this would be relevant if greenlets were adopted as a part of Python?
Worried about monkey-patching? Then only monkey-patch the parts you need to be asynchronous. Worried about magic that you don't understand? Have a look at the code; the magic is actually pretty straightforward after you've paid a little attention to the man behind the curtain.
If you're interested in async stuff for Python, I urge you to have a look at Eventlet or Gevent.
>Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted.
Yeah but - this will be in stdlib. And I think the hope is one-event-loop-to-rule-them-all will let the various frameworks play nice with each other. For instance - Glyph just mentioned to me that he doesn't use IPython any more to work on Twisted code because IPython now has a Tornado event loop which can conflict with the Twisted code he's playing with...
I don't think the hope is that this will be better than twisted or gevent in terms of implementation (obviously the API will be nicer than twisted given Python 3) just that it will be the standard by virtue of being in stdlib.
All the above frameworks will be ported to this and become interoperable - in a similar way to how wsgi works for the web.
Stack-based solutions like gevent, eventlet get screwed over by this.
http://www.tornadoweb.org/en/branch2.3/gen.html
From there, I made a wrapper to wrap potentially blocking logic in threads so I could have code that isn't async/non-blocking behave like it was.
I wasn't a fan of stdlib async until reading these slides.
If you are a fan of this API, try Twisted
Discussions quickly turn theoretical and academic. "But you don't know when your green threads will switch, man, so I'll add yields in there for you". Yes, and then also make sure there is a complete universe of libraries.
Python is awesome not just because it is fun to write little hello world examples in (so is Logo), it is awesome because it is easy to GetShitDone(TM) quickly. The big part of GetShitDone(TM) quickly is reusing libraries not rewriting everything from scratch.
Using an exotic database for some reason -- great. Found a Python library to interact with it -- great. Oh but my framework is based on Deferreds and this one was written without Deferreds or this one returns Futures. Sorry, go write your own from scratch.
This has been the story of my life for 5+ years search or re-writing Twisted version of already existing libraries.
Now at least just adopt Twisted and go with it if they are going this route. But now, they are 'standardizing' on something new. I think had they done this in 2007, yeah rock on, that would have made sense. They didn't. What saved and kept Python on the back-end during the past 5 or so years was greenlet (eventlet and gevent). Guido is kicking all those people in nuts and saying, "no", we'll do Twisted now (with some changes).
Possible solution: "Standardizing gevent solves all its problems".
One of the responses: "I like to write clean code from scratch".
Another: "I really like clean interfaces".
Perhaps the whole thing will make more sense once Guido provides more detail, but I'm underwhelmed and confused.
Since Guido has the barest of descriptions on how this works, you may find the C# async description useful. [1]
[1] http://msdn.microsoft.com/en-us/library/vstudio/hh191443.asp...
@coroutine
def getresp():
s = socket()
yield from loop.sock_connect(s, host, port)
yield from loop.sock_sendall(s, b'xyzzy')
data = yield from loop.sock_recv(s, 100)
# ...
def getresp():
s = socket()
loop.sock_connect(s, host, port).add_done_callback(lambda:
loop.sock_sendall(s, b'xyzzy').add_done_callback(lambda:
data = loop.sock_recv(s, 100).add_done_callback(lambda:
# ...
)
)
)
It is Eventlet and Gevent have that magic. Here is how that looks:
def getresp():
s = socket()
s.connect((host,port))
s.sendall(s,b'xyzzy')
data = s.recv(s,100)
So chains of `yield`s and `yield from`s will bubble to the event loop.
"...run code in another thread - sometimes there is no alternative - eg. getaddrinfo(), database connections"
Just thought I'd mention that async-supporting DNS libs do exist (eg. gevent ships with C-ares), and in particular I've used async postgres database connections in both C and gevent. The code to gevent-ise psycopg2 connections is about 10 or 15 lines, iirc.
Because psycopg2 has supported async OOTB since 2.2 by exposing a pollable socket: http://initd.org/psycopg/docs/advanced.html#asynchronous-sup...
There are limitations though, as noted by the docs: COPY and LOs don't work.
Cool to see languages learning from one another.
I really wonder why that is not the case in Ruby. I mean there are some, but there's mostly confidential and there doesn't seem to be much interest around them. Especially not to the point that the project leader would take a stab at it.
Good on Python anyway, competition is good.
Python's "yield from" hands off execution to a sub-generator: http://www.python.org/dev/peps/pep-0380/
Also, the main point of this is to allow for different async libs to find some common ground to stop the madness of having twisted-specific, tornado-specific, etc... The scientific community does not have this pb because everybody uses numpy.
Async interests potentially everybody, including the server guys, the backend guys AND the scientific community.
Not to mention that this is a few contained classes, whereas the scientific stuff is tons and tons of code to be included into Python, including lots of Fortran and C, that would more than triple the size of the standard library.
Lastly, node.js? Lots of languages have a good story for async, from C# and Scala, to Go and Rust...