Futures, Tasks, and Threads

Understanding when to use threads vs async requires considering their different tradeoffs and capabilities.

Execution Models

Threads:

  • OS-managed, preemptive multitasking
  • Heavy resource overhead (~2-8MB stack per thread)
  • True parallelism on multi-core systems
  • Context switches managed by OS scheduler

Tasks (Async):

  • Runtime-managed, cooperative multitasking
  • Lightweight (~KB overhead per task)
  • Concurrency via yielding at await points
  • Can run millions concurrently

Interchangeable APIs

Many patterns work with both models:

extern crate trpl; // required for mdbook test

use std::{pin::pin, thread, time::Duration};

use trpl::{ReceiverStream, Stream, StreamExt};

fn main() {
    trpl::run(async {
        let messages = get_messages().timeout(Duration::from_millis(200));
        let intervals = get_intervals()
            .map(|count| format!("Interval #{count}"))
            .throttle(Duration::from_millis(500))
            .timeout(Duration::from_secs(10));
        let merged = messages.merge(intervals).take(20);
        let mut stream = pin!(merged);

        while let Some(result) = stream.next().await {
            match result {
                Ok(item) => println!("{item}"),
                Err(reason) => eprintln!("Problem: {reason:?}"),
            }
        }
    });
}

fn get_messages() -> impl Stream<Item = String> {
    let (tx, rx) = trpl::channel();

    trpl::spawn_task(async move {
        let messages = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"];

        for (index, message) in messages.into_iter().enumerate() {
            let time_to_sleep = if index % 2 == 0 { 100 } else { 300 };
            trpl::sleep(Duration::from_millis(time_to_sleep)).await;

            if let Err(send_error) = tx.send(format!("Message: '{message}'")) {
                eprintln!("Cannot send message '{message}': {send_error}");
                break;
            }
        }
    });

    ReceiverStream::new(rx)
}

fn get_intervals() -> impl Stream<Item = u32> {
    let (tx, rx) = trpl::channel();

    // This is *not* `trpl::spawn` but `std::thread::spawn`!
    thread::spawn(move || {
        let mut count = 0;
        loop {
            // Likewise, this is *not* `trpl::sleep` but `std::thread::sleep`!
            thread::sleep(Duration::from_millis(1));
            count += 1;

            if let Err(send_error) = tx.send(count) {
                eprintln!("Could not send interval {count}: {send_error}");
                break;
            };
        }
    });

    ReceiverStream::new(rx)
}

Key differences:

  • thread::spawn vs trpl::spawn_task
  • thread::sleep vs trpl::sleep
  • Both return handles that can be awaited/joined
  • Same resulting streams despite different execution models

Architectural Hierarchy

Runtime (manages tasks)
├── Task 1 (manages futures)
│   ├── Future A
│   ├── Future B
│   └── Future C
├── Task 2 (manages futures)
│   └── Future D
└── OS Threads (work-stealing execution)
    ├── Thread 1 
    ├── Thread 2
    └── Thread 3

Three levels of concurrency:

  1. Futures: Finest granularity, cooperative yield points
  2. Tasks: Group related futures, runtime-scheduled
  3. Threads: OS-scheduled, can run tasks in parallel

Performance Characteristics

Threads excel at:

  • CPU-intensive work (parallel computation)
  • Blocking operations without async alternatives
  • Fire-and-forget background work
  • Simple parallelism requirements

Async excels at:

  • IO-intensive operations (network, file system)
  • High-concurrency scenarios (thousands of connections)
  • Structured concurrency patterns
  • Resource-constrained environments

Hybrid Patterns

Combine both approaches for optimal performance:

extern crate trpl; // for mdbook test

use std::{thread, time::Duration};

fn main() {
    let (tx, mut rx) = trpl::channel();

    thread::spawn(move || {
        for i in 1..11 {
            tx.send(i).unwrap();
            thread::sleep(Duration::from_secs(1));
        }
    });

    trpl::run(async {
        while let Some(message) = rx.recv().await {
            println!("{message}");
        }
    });
}

Pattern: Use threads for blocking operations, async channels for coordination.

Work-Stealing Runtimes

Modern async runtimes use thread pools with work-stealing:

┌─────────────────────────────────────────┐
│ Async Runtime                           │
├─────────────────────────────────────────┤
│ Task Queue: [Task1, Task2, Task3, ...]  │
├─────────────────────────────────────────┤
│ Thread Pool:                            │
│ ├─ Worker Thread 1 ──┐                  │
│ ├─ Worker Thread 2 ──┼─ Work Stealing   │  
│ ├─ Worker Thread 3 ──┘                  │
│ └─ Worker Thread 4                      │
└─────────────────────────────────────────┘

Tasks can migrate between threads for load balancing, combining the benefits of both models.

Decision Framework

Choose Threads when:

  • Heavy CPU computation (image processing, mathematical calculations)
  • Blocking C libraries without async bindings
  • Simple parallel algorithms (embarrassingly parallel problems)
  • Need guaranteed execution progress (real-time systems)

Choose Async when:

  • Network servers handling many connections
  • IO-heavy applications (file processing, database queries)
  • Event-driven architectures
  • Memory-constrained environments

Use Both when:

  • Web applications (async for request handling, threads for background jobs)
  • Data pipelines (async for coordination, threads for CPU processing)
  • Gaming (async for network, threads for physics/rendering)

Cancellation and Cleanup

Threads: Limited cancellation support, manual cleanup

// Thread cancellation is tricky
let handle = thread::spawn(|| {
    // No built-in cancellation mechanism
    loop { /* work */ }
});
// Must use external signaling

Async: Built-in cancellation when futures are dropped

// Automatic cleanup when dropped
let task = spawn_task(async {
    some_operation().await; // Cancelled if task is dropped
});
drop(task); // Automatically cancels and cleans up

Performance Guidelines

Avoid:

  • Creating threads for each IO operation (thread-per-request anti-pattern)
  • Async for CPU-bound work without yield points
  • Mixing sync and async code unnecessarily

Optimize:

  • Use spawn_blocking for sync code in async contexts
  • Pool threads for repeated CPU work
  • Batch async operations to reduce overhead
  • Choose appropriate buffer sizes for channels

Summary

Threads: OS-managed parallelism, higher overhead, preemptive scheduling Tasks: Runtime-managed concurrency, lightweight, cooperative scheduling
Futures: Building blocks for async state machines

The choice isn’t exclusive - modern applications often use both approaches where each excels. Async provides better resource utilization for IO-bound work, while threads provide true parallelism for CPU-bound tasks.

Best practice: Start with async for concurrency needs, add threads for parallelism where beneficial. Use work-stealing runtimes that provide both models efficiently.