Medium
Per-task thread spawn vs a fixed worker pool
A server processes a steady, high-rate stream of small incoming tasks (say, microseconds of actual work each). Version A spawns a brand-new std::thread per task and joins it before handling the next one, as shown. Version B instead maintains a small fixed pool of long-lived worker threads that pull tasks from a shared queue. As the task arrival rate increases, which most accurately describes what happens to Version A?
main.cpp
// Version A: a new OS thread per task, in a hot loop
while (moreTasksArriving()) {
Task task = nextTask();
std::thread t([task] { handle(task); }); // real OS thread spawn every time
t.join();
}
// Version B (conceptually): a fixed pool of N long-lived worker threads
// that block on a shared queue and pull the next task when idle.
// No new OS thread is ever created per task.39 of 51 in Concurrency