Baike.dev
All toolsOnline toolsAI codingTrendingOpen sourceNewsSubmit
Log in
< Back to tools
F

fanout

> 编程语言
Open source

Fanout - make writing parallel code even easier

391 stars0 likes1 views
WebsiteGitHub

About

Fanout - make writing parallel code even easier

Fanout - make writing parallel code even easier

This code is port from sample code of Go blog post Go Concurrency Patterns: Pipelines and cancellation's Bounded parallelism section sample code

I made the fanout pattern sample code reusable, So you can easily write parallel code without worry about fatal error: all goroutines are asleep - deadlock!, Which I encountered quite often when trying to write parallel code, and difficult to figure out why.

From the blog post:

Multiple functions can read from the same channel until that channel is closed; this is called fan-out. This provides a way to distribute work amongst a group of workers to parallelize CPU use and I/O.

For a big list of data, You want to go though each of them and do something with them in parallel. and get the result for each of them to another list for later use. But normally you don't start a new goroutine for every one of them, Because you might don’t know how big the list is, and if it’s too big, that would eat too much memory. This package makes writing this kind of program easier.

API

go
type Worker func(input interface{}) (result interface{}, err error)
func ParallelRun(workerNum int, w Worker, inputs []interface{}) (results []interface{}, err error) {

The package contains these two public api:

  1. Worker is the custom code you want to run,
  2. ParallelRun start to run the Worker
    • workerNum: how many goroutines to start consuming the inputs list, it could be larger than your list size, or smaller, If it’s larger, some of the goroutines will run empty because they can’t get input to work from the channel.
    • inputs: the inputs list that you first need to convert your list to []interface{}
    • results: the result list that returned from the Worker, you normally want to go through them and cast them back to your the real type the Worker actually returns.

A chart to describe how it works

After you fire up the ParallelRun func, It will instantly start the workerNum goroutines, and start to simultaneously work on the Inputs List. After all workers finished without error, It will return the results list. If any of the workers return error. All the other workers will immediately stop and return the first error to ParallelRun

…

Example: check my ideal domain is available or not

For example, I have a text file contains Chinese words like this

小米
翻译
老兵
电影
土豆
豆瓣
文章
圆通
做爱
迅雷
昆明
地图
... and thousands more

I want to do first to convert the word to PinYin, and then suffix .com, Then use whois command to all of them to see if they are still available. So that if they are, I can quickly register it.

The non-parallel slow program will look like this:

…

This would be ridiculously slow, Because for each word, the program start to run command whois, and after it finishes, It start to run next word. It would take days to run for example 30000 words.

Change it to parallel with fanout

You need to install the package if you haven't:

go get github.com/sunfmin/fanout

And with one call, you get it all parallel:

…

The full runnable code is here: https://github.com/sunfmin/domaincheck/blob/master/main.go

Q&A

  1. Does fanout.ParallelRun block execution?

    Yes, it does, It will wait until all the running goroutines finish, then collect all the results into the returned []interface{} value, and you might want to unwrap it and sort it for later use.

  2. How does it make the program run faster?

    For example there is a list contains 20 elements need to process, if the func for processing one elements takes exactly 1 second. In a non-parallel way, It basically will spend 20 seconds to do the work and show you the result of 20 elements. But by using fanout.ParallelRun, if you set the workerNum to be 20, In total, it will only spend the longest execution time 1 second to finish the 20 elements. So it's a 20x improvement. In reality it won't be exactly a 20x improvement. Because of the cores of CPU, and how intensive the program is using I/O. But it will maximize CPU usage and I/O throughput.

Enjoy!

Issues· 0 open

View all issuesOpen on GitHub

No open issues yet, or sync has not completed.

> Tags

Go

No comments yet. Be the first to share.

> Details

PublishedAug 1, 2026
UpdatedSep 17, 2026
Category编程语言
PricingOpen source

> Related tools

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言