Compression improvements using lossy techniques

Author: lrolsenCreated Oct 12, 2020Updated Sep 9, 2026
LabelsColumnstorefeature-request

Hi,

Have you considered implementing any of the improvements that VictoriaMetrics have added to standard Gorilla compression?

They are outlined [here] (https://medium.com/faun/victoriametrics-achieving-better-compression-for-time-series-data-than-gorilla-317bc1f95932)

As a simple test I tried converting float8 to int8 in a table containing 224mio rows of financial data. From this: CREATE TABLE public.trade_double ( instrumentvenueid int4 NOT NULL, externaltradeid int4 NOT NULL, executedatutc timestamp NOT NULL, price int8 NOT NULL, qty int8 NOT NULL );

To this: CREATE TABLE public.trade_int ( instrumentvenueid int4 NOT NULL, externaltradeid int4 NOT NULL, executedatutc timestamp NOT NULL, price int8 NOT NULL, qty int8 NOT NULL );

With the following compression settings: ALTER TABLE trade_double SET ( timescaledb.compress, timescaledb.compress_segmentby = 'instrumentvenueid', timescaledb.compress_orderby = 'executedatutc ASC, externaltradeid' );

The conversion was done by multiplying the price and qty columns by 1000000.

Uncompressed the table was 24GB. Compressed with doubles the size went down to 3347 MB whereas as it went down to 2047 MB with int's. So a 39% reduction with that simple change.