Medium
Stackless coroutines vs one OS thread per connection
A server needs to handle tens of thousands of concurrent, mostly-idle network connections, each spending nearly all of its time waiting on I/O. handleConnection() below is a C++20 stackless coroutine that suspends at co_await points instead of blocking a thread. Compared to a naive one-OS-thread-per-connection design, what is the core reason the coroutine approach scales to far higher connection counts?
main.cpp
Task handleConnection(Socket sock) {
while (auto request = co_await sock.readRequest()) {
auto response = co_await computeResponse(*request);
co_await sock.writeResponse(response);
}
// suspends here at each co_await, giving control back to
// a small pool of OS threads that drive many such coroutines
}
// vs. the naive design: one std::thread per connection, each
// blocked inside a synchronous read() call waiting for I/O.42 of 51 in Concurrency