Originally published on tamiz.pro.
In 2021, a team running a Go-based infrastructure service hit 1 million concurrent goroutines under load.
The lessons they pulled from that scale—structured concurrency, cancellation propagation, resource budgeting, observable failure—landed differently this time around.
Today, the same patterns are surfacing as engineers build production AI agent systems, except instead of goroutines racing on an event loop, we're managing LLM calls, tool executions, and streaming responses across distributed services.
The parallel isn't coincidental.
Both domains share a core tension: unbounded fan-out looks elegant in code and catastrophic in production.
Understanding how the Go community solved this at massive scale gives AI engineers a head start on the problems that are now hitting agent platforms.
1.
The Fan-Out Problem: When Parallelism Becomes Chaos Goroutine Leak at Scale The 1M-goroutine incident started innocently.
A request handler spawned a worker goroutine per downstream call: Under normal load this was fine.
But when the system saw a burst of requests—each with 50–200 subtasks—the goroutine count spiked.
Without cancellation on , every in-flight goroutine survived until its upstream request timed out or the process was killed.
The Go runtime didn't crash (it's designed for this), but the scheduler overhead became significant, and memory from pending operations accumulated.
The fix wasn't removing goroutines—it was adding boundaries: The Agent Equivalent AI agent frameworks exhibit the exact same pattern today.
Consider a typical "plan-and-execute" agent: A single user request can fan out to 20–50 parallel LLM calls, each with its own context window, API latency, and error surface.
Without concurrency limits, you're hitting rate limits, blowing your token budget, and degrading response quality for all concurrent users.
The goroutine leak becomes a token leak and a latency cascade.
The parallel solution is identical: semaphore-based bounded concurrency, context propagation, and structured cleanup: The lesson scales: whether it's goroutines, HTTP connections, or LLM invocations, unconstrained parallelism is a design smell.
The Go community learned this the hard way at scale.
AI engineers are learning it now—often before the hard part.
2.
Cancellation: The Silent Killer of Correctness How Goroutines Handle Abandonment Go's package is a masterclass in cooperative cancellation.
When a parent context is cancelled, all descendants must observe that signal and stop work: The critical invariant: every goroutine in the call tree inherits a context that can be cancelled.
If any link in the chain drops the context and spawns an uncontrolled goroutine, you have a leak.
The Go runtime doesn't enforce this—you do.
What This Means for Agents n AI agent pipelines face the same cancellation problem, but it's harder to see.
When a user cancels a long-running agent, you need to propagate that signal through: In-flight LLM API calls — Must be aborted (or at least ignored when responses arrive) Tool execution goroutines — Must respect cancellation Callback/update streams — Must stop pushing results to dead handlers State mutations — Should be rolled back or abandoned cleanly The insight from the Go community: cancellation isn't an error handling concern—it's a correctness concern.
An agent that continues producing results after the user has moved on isn't just wasteful; it's actively harmful if those results feed back into state that another request reads.
3.
Structured Concurrency: From Channels to Tool Calling The Go Philosophy Go's guiding principle for concurrency is well-known: "Do not communicate by sharing memory; share memory by communicating." Channels enforce ordering, prevent races, and make the control flow explicit.
A goroutine that receives on a channel will block until data arrives or the channel closes—it doesn't spin, poll, or guess.
Each stage communicates through typed channels.
There are no share