undefined | Better HN

0 pointsavianlyric2y ago0 comments

I think there is value in elevating it from an ergonomic point of view, especially when walking that boundary between concurrency and parallelism.

Pure concurrency has the advantage that you can do away with a lot of complex and nuanced synchronism mechanisms, by virtue of the fact you’re not actually sharing memory between parallel lines of computation. Something that makes writing correct concurrent code quite a bit easier and friendlier. In that world have clear and explicit markers of when a function call might result in you yielding to the event loop, and thus memory values you read previously in your function might change, is very handy. Especially if there’s a nice mechanism to delay that yield until after you’ve completed all your important memory operations, and have confidence that your computed values are consistent.

Coroutines are certainly a different approach to the same problem, so hide the blocking nature of functions in a neat way, but at the cost of requiring you to start using those complex synchronisation primitives, because any function call or operation might result in an implicit yield, and thus you can’t predict when memory values might change.

My first introduction to async/await was the Twisted framework for Python. It wasn’t called async/await back then, the principles were identical. Twisted made it possible to write pretty high performance concurrent network code in python, in a way that was very understandable, and _safe_, without resorting to multi-threading or multi-processing. As a result I think the async/await in Python is actually a really good idea. When used correctly, it makes it possible to write really nice, performant, code in python, without resorting to parallelism, and all the pitfalls that come with that (I.e. synchronisation). Async/await provides a nice middle ground between full on parallelism, and single threaded blocking code with no ability to interleave IO operations.

0 comments

gpderetta2y ago

> concurrency has the advantage that you can do away with a lot of complex and nuanced synchronism mechanisms, by virtue of the fact you’re not actually sharing memory between parallel lines of computation.

That's the bit I reject. In practice you are saying that there are no reentrancy concerns between preemption points (async calls), and marking those points explicitly in code help avoid bugs.

I claim that:

a) there can be are reentrancy issues even in regions only performing sync calls (due to invoking callbacks or recursively reentering the even loop)

b) if we value explicit markers for reentrancy, we should be instead explicitly marking reentrancy-unsafe regions with atomic blocks instead of relying on the implicit indirect protection of within-async regions.

With async you still have to think about synchronization, but instead of being explicit and self-documenting in code with synchronized objects and critical sections, you have to rely on the implicit synchronization properties. And if you write code that rely on it, how do you protect against that code breaking when someone (including ourselves) shuffles some async calls around in three months?

In fact something like rust, thanks to lifetimes and mutable-xor-shared, has much better tools to prevent accidental mutation.

Don't get me started on python, asyncio is terrible in its own unique ways (the structured concurrency in trio and similar seems much saner, but I have yet to use).

[Sorry for continuing this argument, as you can tell I have quite strong opinions on it; feel free to ignore me if you are not interested]

avianlyricOP2y ago

> there can be are reentrancy issues even in regions only performing sync calls (due to invoking callbacks or recursively reentering the even loop)

You would hope if this was done properly you wouldn’t be using callbacks at all, because that’s kinda throwing away any benefits async/await provides, and reentrancy to the event loop should require explicit markings.

> if we value explicit markers for reentrancy, we should be instead explicitly marking reentrancy-unsafe regions with atomic blocks instead of relying on the implicit indirect protection of within-async regions.

In principle yeah kinda agree, but a lot of code isn’t reentrancy-safe, I would argue that most code isn’t reentrancy-safe, unless carefully designed to be reentrancy-safe. So a programming model that implicitly makes most code protected against reentrancy does provide value, and make it harder for difficult to debug concurrency bugs to slip in.

I don’t necessarily think it’s the “best” approach, I much prefer people actually think about their code carefully, and be explicit with their intentions. But that requires quite a lot a of experience, and understanding the detailed nuances that come with parallelism, something many engineers simply don’t have. So I think there’s a lot of value in programming paradigms that provide additional protection against those types of errors, but without forcing the type of rigour that Rust does, due to the learning barrier it creates.

I suspect that async/await is here to stay for now, but I very much see it as part of a continuum of concurrency paradigms that we’ll eventually move past, once we find better ways of writing safe concurrent code. But I suspect we’ll only really discover those better ways once we fully explored what async/await offers, and completely understand the tradeoffs it forces.

j / k navigate · click thread line to collapse