Faster way to generate local dataset
Author: zeusdeuxCreated Oct 19, 2025Updated Apr 9, 2026
@ctbur and I wanted to tackle this challenge but got frustrated waiting ~10+ min for the local file to generate.
For anyone that's interested, here's a version of the data generation script that runs in ~1min on my i7 + SSD from 2020 — https://gist.github.com/zeusdeux/307cf43130a8071d25f26a50d1c6a66d
Copy it locally and go run main.go to execute. Go itself is just a binary download so no Java sdk install shenanigans and no fighting maven.
Source: gunnarmorling/1brc