Reconsider the query log format

Author: ainar-gCreated Nov 10, 2020Updated Sep 16, 2026
Labelsenhancementresearch

The current way in which we handle query logs has several issues: it's racy, files can get pretty large, the search algorithm is too complex, and we can only do basic filtering and very little aggregation. But we also don't have a lot of alternatives. Ideally we would like to have something like an SQL database, à la SQLite, but that would require cgo, which comes with its own bag of issues. Package modernc.org/sqlite seems interesting, but it's a work-in-progress and currently doesn't support all our platforms.

I will update the issue when we'll think of something better. Perhaps SQLite is not completely out of the question if we find a way to cross-compile it without too much pain for the developers.