Optimux is a Go service that resizes, re-encodes, and streams images and video on the fly — , done.
It's been running as an internal service for a while.
I'm open-sourcing it now, AGPL-3.0, because the worker pool underneath it — the part that decides how many goroutines are doing image processing at any given moment — ended up being the most interesting thing in the codebase, and it deserves to exist somewhere other than a private repo.
This is the story of how that worker pool got there, including the part where the pattern everyone starts with turned out to be actively wrong for this workload.
Starting point: the Job/Dispatcher pattern Like a lot of people who've had to build a Go job queue, I started from Marcio Castilho's Handling 1 Million Requests per Minute with Golang.
Worth being precise about what that article actually builds, because the mismatch with image processing turned out to matter a lot: A struct wraps one unit of work (there, a POST payload headed to S3).
A buffered receives incoming jobs.
A owns a — a channel of channels.
Each has its own , and the moment a worker finishes a job, it pushes its own channel back into the shared .
The dispatcher's loop just pulls one worker-channel off and hands it the next job.
No worker ever polls for work or reports "I'm free" via a flag — availability is "my channel is currently sitting in the pool." and are env-configured, fixed for the process lifetime.
The article's numbers are genuinely good: they took a system that needed ~100 EC2 instances down to 4 instances handling close to a million requests a minute, by replacing unbounded goroutine spawning with this bounded pool.
But the workload is uploading JSON payloads to S3 — almost entirely I/O wait, barely any CPU.
I built the same pattern, pointed it at image resizing instead, and it did not translate.
The pattern didn't fit, and pprof proved it I wired up the Job/Dispatcher pattern for image jobs and it was slow in a way that didn't make sense from the throughput numbers alone.
So I pulled a blocking profile, and it was unambiguous: pprof blocking profile shows 67.99% of total time spent waiting on .
That's workers blocked waiting for jobs, and the dispatcher blocked waiting for workers to hand their channel back — a pattern built for a workload where a worker is cheap and fast per unit (fire an S3 PUT, wait on the network, done) turns into mostly idle channel choreography when a unit of work is hundreds of milliseconds of CPU-bound image processing instead.
The channel-of-channels handoff, the dispatcher loop, the per-job registration — all of that overhead is invisible when a worker's actual job takes microseconds of your attention and the rest is network wait.
It stops being invisible when the job itself is the bottleneck.
Target throughput was ; no matter what I tuned in this shape, it sat at .
So I threw it out.
Not tuned, not patched — removed, and rebuilt from a synchronous baseline up, one variable at a time, so I'd actually know what each change bought me instead of tuning inside a pattern that was wrong from the start.
Rebuilding from zero, one variable at a time Synchronous baseline, no workers at all — inline resize, respond directly. .
Already ahead of the dispatcher pattern's , which was the first sign the problem wasn't "not enough workers," it was the shape of the concurrency itself.
Single channel, single worker.
The smallest thing that still queues — one goroutine pulling off one channel, closer to a single core churning a work list than to a "pool" in any real sense. — worse than doing nothing concurrent at all.
Concurrency has a floor cost, and this workload was paying it without buying anything back yet.
Single queue, 4 workers. , average latency .
First real win, and a small one.
Two queues — fetch and process, split apart — 4 workers each. , marginally lower throughput than the single queue, but the latency distribution across percentiles visibly smoothed out.
Tracing explained why: fetching a source image from tmpfs cost ; processing it (libvips, resizing a 3.1MB source down to a 120×240 webp) cost .
Those are two stages with wildly different service times sharing a queue and worker pool — an impedance mismatch, where a burst of cheap fetches can queue up behind a slow processor, or the reverse.
Two queues, but weighted 2 fetchers / 4 processors, leaning into the imbalance on purpose.
Performed worse than the even split, still around .
My read at the time: this isn't a batch pipeline where you can freely over-provision the expensive stage and let a queue absorb the mismatch — it's a real-time request path where the caller is still on the other end of the HTTP connection waiting, so over-provisioning one stage just moves the queueing, it doesn't remove it.
I also tried a demand-driven producer/consumer setup along the way — closer to Elixir's GenStage, where the consumer explicitly asks the producer for N items instead of the producer pushing whenever it has something.
It didn't pan out for this workload, and honestly the specifics didn't survive in my notes — only the conclusion did: the demand/ack round-trip was adding coordination cost that a plain buffered channel already got for free, without a corresponding improvement in how work actually got scheduled.
Separately, before any dynamic scaler existed, I added something orthogonal to worker count entirely: instead of buffering the whole encoded image and writing it in one response, the handler started flushing early and streaming bytes out as the encoder produced them.
That alone moved the needle independent of worker shape — , latency down to .
What actually generalizes, and what doesn't Here's the actual lesson, not just the numbers: the Job/Dispatcher pattern isn't wrong, it's scoped to a specific kind of workload — one where the external resource (S3, a network call, a database) is the thing you're rationing, and your own CPU is basically idle waiting on it.
In that world, a large fixed worker count is nearly free — you're just capping how many outstanding waits you allow. libvips-backed image processing inverts that: the constrained resource is your own CPU and memory, fetch and process have genuinely different cost profiles, and a fixed picked once has no way to track a queue that swings between bursty and empty. on the EC2 box backed this up directly — CPU usage swinging from 80%+ down to 40% inside the same short window, spiking as high as 8.74% (hypervisor-stolen cycles, which no amount of worker retuning fixes).
A workload like that needs the worker count itself to be a live variable, not a constant — which is what actually motivated moving off any fixed-size pool entirely, GenStage-shaped or otherwise, toward something that watches queue depth and scales.
It's worth being precise about where the two patterns actually diverge structurally, because it's not just "one has a scaler and one doesn't." The dispatcher pattern's availability signal is a second layer of indirection: each worker owns a private , and "I'm free" is expressed by pushing that channel into a shared .
That's channels for workers — one pool channel plus one per-worker channel — and a dispatcher goroutine whose whole job is shuttling a channel out of the pool, handing it a job, and waiting for it to come back.
It's a clean pattern, but it exists to solve a problem optimux's worker pool doesn't have: every worker in pulls directly off one shared .
There's no dispatcher goroutine, no per-worker channel, no explicit "I'm free" message at all — a worker's availability is nothing more than "currently blocked on a receive from ," which Go's own runtime already arbitrates correctly among however many goroutines are competing to receive.
The single-shared-queue shape showed up as early as the "single queue, 4 workers" experiment above, well before any scaler existed, and every version since kept it — the channel-of-channels indirection never came back, because a single channel with competing r