Why Async/Await Fails Concurrency and the Tokio Rayon Trap
The Tokio/Rayon Trap and Why Async/Await Fails Concurrency
I argue that async/await makes concurrency easy to write but complex to operate by hiding structural chaos. By confusing asynchrony with concurrency, it forces developers to manually manage runtimes like Tokio and Rayon, turning them into human schedulers. I propose Project Tina, a deterministic framework that replaces runtime magic with strict architectural guarantees to ensure predictability over brevity.
Predictability beats brevity.
- FridgeSeal
This reads an awful lot like the prompter had a semi-confusing time with some async, and had their favourite model write an upset blog post about it.
I don't think these systems are perfect, nor are they fit for every use-case, but some of the complaints ring a bit hollow.
> fetching a database record over the network, then immediately crunching the data. But what happens when that data crunching involves parsing a 10MB JSON payload
Mixing IO-sensitive code and blocking code causes issues, who knew? I'm not quite sure how these libraries are supposed to magically _save_ you from this?
> When these latency spikes occur, the answer is always the same: separate your runtimes.
Well yeah. "Why doesn't my daily-driver cut sick lap times around the Nurburgring?" If you want to run blocking, non-interleaved code, don't do it in an executor expecting small, interleaved, non-blocking tasks. The docs for Tokio even mention this, and provide a number of worked examples of integrating/bridging sync and async code.
> If a developer must manually partition I/O and compute, strictly police the boundaries to prevent deadlocks, and ferry data between two different runtimes with two different mental models, the async abstraction has failed.
Not necessarily. If I'm chasing a performance target, and the tool gets me 80-90% of the way there, to the point where my next task is optimising layout and caching, I call that a win. That's performance ground we'd have to address at some point if we want t […]
- elendilm
<When WhatsApp pushed the Erlang BEAM virtual machine to its limits on 100+ core machines, the system choked. As detailed by Robin Morisset, idle threads trying to steal work spent all their CPU cycles fighting over the global runq_lock5>
A simple burst of memmap + soft fault with 100 or 1000 threads on a normal laptop would tell you that thread contention is real and cache locality gets destroyed. Couple that with pinned threads. You can see the latency increase by increasing thread count. Add to that the motherboard interconnect tax for numa systems. Work stealing is not the way for increasingly many workloads on modern hardware.
Recently we built Dip, our in-house ephemeral + parallel database, and we went with may coroutines + work pinning to the same thread which also nicely becomes numa aware via architecture.
Increasing threads beyond system's hardware cores/threads resulted only in marginal gains of a couple of milliseconds worth of differences on huge workload with large increase in memory (thread stack) used by the massive number of threads.
- Animats
There are some fundamental assumptions in the Rust async system:
- The program is mostly I/O bound.
- All tasks have equal priority.
If your program isn't like that, the Tokio model is a bad match to the problem.
Real time control is not like that. MMO and metaverse game programs are not like that. Most web stuff is, but that's a special case. A big special case, but a special case.
- haberman
I see two solid points here:
1. It's not reasonable to expect the application layer to carefully partition its work into "I/O heavy" and "CPU heavy" parts.
2. It's not reasonable to queue up an arbitrary amount of work without back-pressure.
I haven't used Tokio much, but if it falls prey to these pitfalls, it would make me pause before adopting it.
I think there are probably ways of using Rust async that don't fall prey to these. Maybe not so much with network servers (I haven't written that many of those), but models where you are evaluating a graph and have more control over how new work is added to the system.
- oersted
I was looking forward to checking out Project Tina until I realised it was a completely different language. Classic story. Surely you can build a thread-per-core message passing concurrency framework in Rust, the language is designed to allow such alternatives.
Is Project Tina a bit like the Actor Model, but having actors pinned to cores?
And I don't understand how Tina deals better with the problem of compute-heavy tasks blocking the thread. It looks to me like it is also cooperative concurrency per core, and if one Isolate runs for a long time the other Isolates in that core will not be able to handle their messages.