Async Runtimes & Tokio

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Rust 1.99.0, tokio 1.53 and tokio-util 0.7, checked on 2026-10-06. Rust’s async syntax produces futures, but something has to poll them, wait for I/O and run timers: an async runtime. This page explains what a runtime does and covers tokio, the runtime that axum, hyper, tonic, reqwest and sqlx are built on. It is the shared tokio primer for the Axum section, which links here instead of repeating it.

Executors, reactors and why std has no runtime

A runtime has two halves:

  • an executor, which keeps a queue of tasks (top-level futures) and polls the ones that are ready;

  • a reactor, which asks the operating system (epoll on Linux, kqueue on macOS/BSD, IOCP on Windows) to report when sockets are readable or writable and when timers expire, and calls the corresponding wakers.

The standard library defines Future, Waker and Context but deliberately ships no runtime: embedded systems, browsers (WebAssembly), servers and desktop applications need very different trade-offs. tokio is the dominant general-purpose choice; smol is a smaller alternative, and embassy targets microcontrollers. Futures are mostly portable between runtimes, but I/O types and timers are not — a tokio::net::TcpStream needs the tokio reactor.

Source: Async Book: Runtimes

Starting tokio

[dependencies]
tokio = { version = "1.53", features = ["full"] }   # or pick: rt-multi-thread, macros, net, time, sync, fs, signal
tokio-util = { version = "0.7", features = ["rt"] }  # CancellationToken; "rt" adds TaskTracker

Source: tokio: Feature flags

#[tokio::main] turns async fn main into a normal main that builds a runtime and blocks on the future. The default is the multi-thread scheduler with one worker per CPU core; flavor = "current_thread" runs everything on the calling thread, which suits tests, CLIs and small tools. The builder gives full control:

fn main() -> std::io::Result<()> {
    let runtime = tokio::runtime::Builder::new_multi_thread()
        .worker_threads(4)
        .thread_name("shelf-worker")
        .enable_all()                      // I/O driver and timers
        .build()?;
    let answer = runtime.block_on(async {
        tokio::time::sleep(std::time::Duration::from_millis(10)).await;
        42
    });
    println!("{answer}");
    Ok(())
}

Source: tokio::runtime

The multi-thread scheduler keeps a local run queue per worker and uses work stealing: an idle worker takes tasks from a busy one. I/O readiness and timer expiry wake tasks, which go back on a run queue. Blocking work runs on a separate pool:

Diagram of the tokio multi-thread runtime. A global injection queue receives tasks spawned from outside the workers. Several worker threads, one per core, each have a local run queue and a LIFO slot for a task just woken, and poll one task at a time until it returns Pending or Ready; an idle worker steals half of another worker’s queue or takes from the global queue. A task that blocks a worker stalls every task queued on it. The I/O driver and timer wheel call the wakers of waiting tasks when sockets are ready or timers expire. A separate blocking pool of up to 512 threads runs spawn_blocking closures, tokio::fs and DNS lookups.

Source: Tokio tutorial

Spawning tasks

tokio::spawn(future) submits a future as a new task that runs concurrently (and, on the multi-thread scheduler, possibly in parallel) and returns a JoinHandle<T>. Because the task may run on any worker and outlive the caller, the future must be Send + 'static: it must own its data (use async move, clone Arcs) and must not hold non-Send values across .await.

use std::sync::Arc;
use tokio::task::JoinSet;

async fn fetch_rating(isbn: String) -> (String, u8) {
    tokio::time::sleep(std::time::Duration::from_millis(10)).await;
    let r = (isbn.len() % 5 + 1) as u8;
    (isbn, r)
}

#[tokio::main]
async fn main() {
    let catalog = Arc::new(vec!["9780441013593".to_string(), "9780141439518".to_string(), "9780807083697".to_string()]);

    // One task, awaited through its JoinHandle.
    let first = tokio::spawn(fetch_rating(catalog[0].clone()));
    println!("{:?}", first.await.expect("task panicked"));

    // Many tasks, collected as they finish.
    let mut set = JoinSet::new();
    for isbn in catalog.iter().cloned() {
        set.spawn(fetch_rating(isbn));
    }
    while let Some(res) = set.join_next().await {
        let (isbn, rating) = res.expect("task panicked");
        println!("{isbn}: {rating}");
    }
}

Source: tokio::task::JoinSet

A JoinHandle that is dropped detaches its task (it keeps running); handle.abort() cancels it. A JoinSet aborts all its remaining tasks when dropped, which makes it a good tool for structured concurrency (below). A panic inside a task is caught by tokio and returned as Err(JoinError) from .await on the handle.

Blocking in async code

A worker thread can only poll one task at a time. If a task performs a blocking operation — a CPU-heavy loop, password hashing, std::thread::sleep, synchronous file or network I/O, a blocking database driver — that worker cannot run any other task until it finishes. Every task queued on that worker stalls; with current_thread, the whole program stalls. The symptoms in production are latency spikes and timeouts with low CPU usage; Gangappa & Rangaraj (Mastering Distributed Observability in Rust, chapters 2 and 10) show how this task starvation appears in traces and in tokio-console.

The rule of thumb from the tokio documentation is that a task should not go more than about 10-100 microseconds between .await points. For anything slower:

  • tokio::task::spawn_blocking(closure) runs the closure on the blocking pool and returns a JoinHandle to await;

  • tokio::task::block_in_place (multi-thread runtime only) converts the current worker into a blocking thread;

  • for sustained CPU-bound parallel work, use Rayon (see Data Parallelism with Rayon) or a dedicated thread pool, and communicate with channels;

  • use async APIs for I/O: tokio::fs, tokio::net, async database drivers.

use std::time::{Duration, Instant};

fn expensive_hash(password: &str) -> u64 {
    // stand-in for argon2: deliberately slow CPU work
    let mut h = 1469598103934665603u64;
    for _ in 0..3_000_000 {
        for b in password.bytes() {
            h = (h ^ u64::from(b)).wrapping_mul(1099511628211);
        }
    }
    h
}

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let ticker = tokio::spawn(async {
        let start = Instant::now();
        for _ in 0..3 {
            tokio::time::sleep(Duration::from_millis(10)).await;
            println!("tick at {} ms", start.elapsed().as_millis());
        }
    });

    // Wrong: expensive_hash("hunter2") here would freeze the ticker until it returns.
    let hash = tokio::task::spawn_blocking(|| expensive_hash("hunter2")) // right: off the async worker
        .await
        .expect("hashing task panicked");
    println!("hash {hash:x}");
    ticker.await.unwrap();
}

Source: tokio::task::spawn_blocking

When the blocking closure should be traced as part of the request, carry the current tracing span into it (let span = tracing::Span::current(); spawn_blocking(move || span.in_scope(|| ...))); see Axum: Performance Tuning.

Timers

tokio::time provides sleep, sleep_until, interval (a periodic tick that by default catches up after delays), timeout(duration, future) and Instant. timeout cancels the inner future when the deadline passes.

use std::time::Duration;
use tokio::time::{interval, timeout, MissedTickBehavior};

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let mut reminders = interval(Duration::from_millis(20));
    reminders.set_missed_tick_behavior(MissedTickBehavior::Delay);
    for i in 0..3 {
        reminders.tick().await; // the first tick completes immediately
        println!("send due-date reminders, round {i}");
    }
    let slow = tokio::time::sleep(Duration::from_secs(5));
    println!("{:?}", timeout(Duration::from_millis(30), slow).await.is_err()); // true: timed out
}

Source: tokio::time

tokio::sync primitives

Primitive Use

mpsc::channel(n)

bounded multi-producer, single-consumer queue with backpressure (send().await waits when full); the usual way to feed a worker task

oneshot::channel()

a single value from one task to another: request/response with a worker

broadcast::channel(n)

every receiver gets every message (slow receivers miss old ones with Lagged): fan-out such as a live review feed

watch::channel(v)

the latest value of something that changes (configuration, a shutdown flag); receivers see only the newest

Mutex<T>, RwLock<T>

async-aware locks whose guards can be held across .await

Semaphore

limit concurrency (at most N outgoing requests, N database connections)

Notify

wake one or all waiting tasks without data

 
use tokio::sync::{broadcast, mpsc, oneshot, watch, Semaphore};
use std::sync::Arc;

#[derive(Debug)]
enum Command {
    Lend { isbn: String, reply: oneshot::Sender<bool> },
}

#[tokio::main]
async fn main() {
    // A worker task owns the state; others talk to it through a channel (the actor pattern).
    let (tx, mut rx) = mpsc::channel::<Command>(32);
    let (events, _) = broadcast::channel::<String>(16);
    let events_for_worker = events.clone();
    tokio::spawn(async move {
        let mut lent = std::collections::HashSet::new();
        while let Some(Command::Lend { isbn, reply }) = rx.recv().await {
            let ok = lent.insert(isbn.clone());
            if ok {
                let _ = events_for_worker.send(format!("lent {isbn}"));
            }
            let _ = reply.send(ok);
        }
    });

    let mut feed = events.subscribe();
    let (reply_tx, reply_rx) = oneshot::channel();
    tx.send(Command::Lend { isbn: "978".into(), reply: reply_tx }).await.unwrap();
    println!("lent: {}  event: {}", reply_rx.await.unwrap(), feed.recv().await.unwrap());

    let (config_tx, config_rx) = watch::channel(5u32); // max loans per member
    config_tx.send(8).unwrap();
    println!("current limit {}", *config_rx.borrow());

    let permits = Arc::new(Semaphore::new(2)); // at most two concurrent outbound calls
    let _p = permits.acquire().await.unwrap();
    println!("{} permits left", permits.available_permits());
}

Source: tokio::sync

In the actor pattern above, the state (lent) is owned by one task and never shared, so no lock is needed — often the cleanest design for mutable state in async code. Note that a broadcast::Sender is cheap to clone and already thread-safe; it never needs to be wrapped in Arc<Mutex<...>>.

std::sync::Mutex or tokio::sync::Mutex?

The tokio documentation is explicit: prefer std::sync::Mutex (or parking_lot::Mutex) for data, and use tokio::sync::Mutex only when you must hold the guard across an .await — typically to serialise access to an I/O resource such as a database connection. A std mutex is faster, and as long as the guard is dropped before any .await, blocking on it is brief. The tokio mutex is slower because its lock() is itself async.

use std::collections::HashMap;
use std::sync::{Arc, Mutex};

#[derive(Clone, Default)]
struct Cache {
    titles: Arc<Mutex<HashMap<String, String>>>, // std Mutex: guard never crosses an .await
}

impl Cache {
    async fn title(&self, isbn: &str) -> String {
        if let Some(t) = self.titles.lock().unwrap().get(isbn).cloned() {
            return t; // guard dropped at the end of the `if let` (2024 edition)
        }
        let fetched = fetch(isbn).await; // no lock held while waiting
        self.titles.lock().unwrap().insert(isbn.to_owned(), fetched.clone());
        fetched
    }
}

async fn fetch(isbn: &str) -> String {
    tokio::time::sleep(std::time::Duration::from_millis(5)).await;
    format!("title for {isbn}")
}

#[tokio::main]
async fn main() {
    let cache = Cache::default();
    let c2 = cache.clone();
    let t = tokio::spawn(async move { c2.title("978").await }); // must be Send: it is
    println!("{} / {}", t.await.unwrap(), cache.title("978").await);
}

Source: tokio::sync::Mutex: Which kind of mutex should you use?

Holding a std::sync::MutexGuard across an .await makes the future non-Send, so tokio::spawn rejects it at compile time ("future cannot be sent between threads safely") — a helpful guard rail. Gangappa & Rangaraj (chapter 2) recommend tokio::sync::Mutex as the default for shared state; the tokio documentation recommends the opposite, as above.

Cancellation and structured concurrency

Async tasks are cancelled by dropping their future or calling abort(). Coordinating a graceful stop across many tasks is easier with a CancellationToken from tokio-util: clone it into every task, select! on token.cancelled(), and call token.cancel() once. TaskTracker (also tokio-util) waits for all tracked tasks to finish.

Structured concurrency means tasks do not outlive the scope that started them, so errors and cancellation propagate and nothing leaks in the background. Rust does not enforce it for tokio::spawn, but JoinSet (aborts on drop), join!/ try_join! (no separate tasks at all) and CancellationToken + TaskTracker get close.

use std::time::Duration;
use tokio_util::sync::CancellationToken;
use tokio_util::task::TaskTracker;

#[tokio::main]
async fn main() {
    let token = CancellationToken::new();
    let tracker = TaskTracker::new();

    for id in 1..=2 {
        let token = token.clone();
        tracker.spawn(async move {
            let mut rounds = 0;
            loop {
                tokio::select! {
                    _ = token.cancelled() => break,
                    _ = tokio::time::sleep(Duration::from_millis(10)) => rounds += 1,
                }
            }
            println!("worker {id} stopped after {rounds} rounds");
        });
    }

    tokio::time::sleep(Duration::from_millis(35)).await;
    token.cancel();          // ask everyone to stop
    tracker.close();
    tracker.wait().await;    // and wait until they have
    println!("all workers stopped");
}

Source: tokio_util::sync::CancellationToken

The axum section uses exactly this pattern for graceful shutdown (see Axum: Graceful Shutdown).

Source: Async Book: Structured concurrency

tokio-console

tokio-console is a top-like debugger for async programs: it lists every task with how long it has been polled, idle and busy, warns about tasks that poll for too long (blocking) or that never yield, and shows resources such as semaphores and timers. Enable it with the console-subscriber crate and building with RUSTFLAGS="--cfg tokio_unstable", then run the tokio-console CLI against the process. It is the quickest way to find the blocking calls described above.

Source: tokio-console

Common errors

Error Cause and fix

E0277 "future cannot be sent between threads safely"

a non-Send value (Rc, RefCell borrow, std MutexGuard) lives across an .await in a spawned future: drop it before the .await, restructure, or use Arc/tokio types

E0373 / "borrowed value does not live long enough" in tokio::spawn

the task must be 'static: async move and own or Arc the data

panic "there is no reactor running"

a tokio I/O type or timer used outside a tokio runtime (or from another runtime)

panic "Cannot start a runtime from within a runtime"

calling block_on inside async code; .await instead

latency spikes, starved tasks

blocking work on a worker thread: spawn_blocking

Where the books are outdated

Gangappa & Rangaraj (chapter 2) recommend tokio::sync::Mutex as the default for shared state; tokio’s own documentation prefers std::sync::Mutex unless the guard is held across .await. Yoon (Beginning Axum, chapter 5) wraps a broadcast::Sender in Arc<Mutex<...>>, which is unnecessary because the sender is already Clone + Send
Sync. Both books use tokio 1.4x; the APIs shown here are unchanged across tokio 1.x.

References