`min`/`max` on a `DOUBLE` column is ~6x slower than on `BIGINT`, and even ~9x slower in 2.0
What happens?
min()/max() over a DOUBLE column is far slower than I'd expect. On 100M rows:
min(DOUBLE)is ~6x slower thanmin(BIGINT).min(DOUBLE)is ~3x slower thansum(DOUBLE)on the same column, so it is not the cost of reading the column.
This looks like the floating-point min/max aggregate does not vectorize, while integer min/max and floating-point sum do.
Observed (best of 5, 8 threads, Apple Silicon, macOS)
DuckDB 1.5.5 (stable)
| query | time |
|---|---|
min(i64) |
12.4 ms |
min(f64) |
77.0 ms |
sum(f64) |
27.9 ms |
count(*) |
2.9 ms |
DuckDB 2.0.0 preview (2.0.0.dev2609121639) — same machine, same script
| query | time |
|---|---|
min(i64) |
12.6 ms |
min(f64) |
113.1 ms |
sum(f64) |
17.8 ms |
count(*) |
0.4 ms |
min(f64)costs ~6xmin(i64)and ~3-6xsum(f64), even though all read 100M values andsumtouches the identicalDOUBLEcolumn.- On the 2.0 preview the
DOUBLEminpath is ~1.5x slower again (77 -> 113 ms), whilemin(i64)andsum(f64)are unchanged or faster.
I also observed that the gap is not influenced by value width: FLOAT min is as slow as DOUBLE min despite being half the bytes.
Expected
min/max on DOUBLE to be close to sum on DOUBLE (both are a single linear reduction), i.e. in the tens of ms, not ~6x the integer min. And I would certainly not expect a ~1.5x slowdown on duckdb 2.0.
Environment
- DuckDB 1.5.5 (stable) and 2.0.0.dev2609121639 (preview), Python client (
pip install duckdb). - Python 3.14, macOS (Apple Silicon), 8 threads (
SET threads = 8). - Also reproducible from the
duckdbCLI with the SQL provided in the "To Reproduce" section.
A self-contained Python version of the script is attached below.
repro_float_minmax.pyimport time
import duckdb
N = 100_000_000
con = duckdb.connect(":memory:")
con.execute("SET threads = 8")
con.execute(f"""
CREATE TABLE t AS
SELECT r AS i64, r::DOUBLE + 0.5 AS f64
FROM range({N}) AS _(r)
""")
def timeit(sql, warmup=2, repeats=5):
for _ in range(warmup):
con.execute(sql).fetchone()
times = []
for _ in range(repeats):
t0 = time.perf_counter()
con.execute(sql).fetchone()
times.append((time.perf_counter() - t0) * 1e3)
return min(times)
print(f"duckdb {duckdb.__version__}, {N:,} rows, 8 threads (best of 5, ms)\n")
for label, sql in [
("min(i64) [BIGINT]", "SELECT min(i64) FROM t"),
("min(f64) [DOUBLE]", "SELECT min(f64) FROM t"),
("sum(f64) [DOUBLE]", "SELECT sum(f64) FROM t"),
("count(*) ", "SELECT count(*) FROM t"),
]:
print(f" {label} {timeit(sql):7.1f} ms")To Reproduce
CREATE TABLE t AS
SELECT r AS i64, r::DOUBLE + 0.5 AS f64
FROM range(100000000) AS _(r);
.timer on
SELECT min(i64) FROM t; -- BIGINT min
SELECT min(f64) FROM t; -- DOUBLE min
SELECT sum(f64) FROM t; -- DOUBLE sum, same column
SELECT count(*) FROM t; -- framework floorOS:
aarch64 - macOS 26.6.2
DuckDB Version:
1.5.5 and 2.0.0.dev2609121639
DuckDB Client:
Python and CLI
Hardware:
No response
Full Name:
Jeroen Van Der Donckt
Affiliation:
Flex Analytics
Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?
- Yes, I have
Did you include all code required to reproduce the issue?
- Yes, I have
Did you include all relevant data sets for reproducing the issue?
No - Other reason (please specify in the issue body)
Source: duckdb/duckdb