[Bug] SUM() over DECIMAL silently returns a wrong value on overflow, while check_overflow_for_decimal errors on the same overflow in binary arithmetic
Search before asking
- I had searched in the issues and found no similar issues.
Version
Built from master at commit c4dee4bd5e82c6f0f380e0243cbaf265999e6a6a
(2026-09-17). ./build.sh --fe --be, single FE + single BE, default fe.conf
and be.conf.
Session defaults, unchanged: check_overflow_for_decimal = 1,
enable_decimal256 = 0.
What's Wrong?
SUM() over a DECIMAL column returns a silently wrong value once the
running total leaves the representable range. Doris already detects the very same
overflow in binary arithmetic and refuses the query; the aggregate does not check
at all.
On two rows of DECIMAL(38,0) holding 10^38 - 1:
| expression | result |
|---|---|
a + a |
errors — [E-255]Arithmetic overflow: 99999999999999999999999999999999999999 add 99999... |
a * 2 |
errors — [E-255]Arithmetic overflow: ... multiply 2 = 1999999... |
SUM(a) |
-14028236692093846346337460743176821145 — no error |
The same overflow, on the same two values, in the same query: refused in one expression and answered with a wrong number in the other.
There are two distinct silent regimes, and the first one is the dangerous one
With an ordinary 34-digit per-row value of 10^33, summing n rows:
| n | exact total | returned | |
|---|---|---|---|
| 99 999 | 99999000000000000000000000000000000000 | same | ok |
| 100 000 | 100000000000000000000000000000000000000 | 10000000000000000000000000000000000000 | exactly one tenth — positive and plausible |
| 170 141 | 170141000000000000000000000000000000000 | 17014100000000000000000000000000000000 | one tenth |
| 170 142 | 170142000000000000000000000000000000000 | -17014036692093846346337460743176821145 | wrapped negative |
| 199 998 | 199998000000000000000000000000000000000 | -14028436692093846346337460743176821145 | wrapped negative |
The two boundaries are exactly the two representable limits involved:
- n = 100 000 is where the total first exceeds
DECIMAL(38,0)'s maximum (10^38 - 1). From here the answer is the true total divided by ten — positive, 38 digits, entirely plausible, and wrong by an order of magnitude. - n = 170 142 is where the total first exceeds
int128max (170141183460469231731687303715884105727). From here it wraps negative.
A negative sum of positive values is at least obviously broken. A result that is one tenth of the truth is not — nothing in the result, the type, or the log says anything is wrong.
Every accumulation path is affected, including one that persists to disk
Two rows of DECIMAL(38,0) holding 10^38 - 1; exact total
199999999999999999999999999999999999998:
| path | returned | |
|---|---|---|
SUM(a) |
-14028236692093846346337460743176821145 | wrong |
SUM(a) ... GROUP BY g |
-14028236692093846346337460743176821145 | wrong |
SUM(a) OVER () |
-14028236692093846346337460743176821145 | wrong |
AVG(a) |
8026480282232511287371606174911062.6544 | wrong |
AGGREGATE KEY table, a DECIMAL(38,0) SUM |
-14028236692093846346337460743176821145 | wrong, and stored on disk |
SUM(a) + 0 |
errors | refuses |
SUM(DISTINCT a) |
99999999999999999999999999999999999999 | correct |
Three things worth separating out:
AVG breaks an invariant that needs no arithmetic to check. The average of
two identical values is that value. Both inputs are 10^38 - 1; the answer comes
back as about 8.03 x 10^33. AVG is dividing the already-wrapped total.
The AGGREGATE KEY case is not a query-time artifact. That table
pre-aggregates on write, so the wrapped value is what the storage engine
persists. Two inserts of 10^38 - 1 leave the table holding a negative number
and the inputs are gone.
SUM(a) + 0 errors, and the error quotes the corrupted value:
[E-255]Arithmetic overflow: -14028236692093846346337460743176821145 add 0 = DECIMAL128I ...So the aggregate has already produced the wrapped number and handed it downstream; the only reason anything complains is that the binary operator is checked. This localises the gap precisely to the aggregate's own accumulation.
enable_decimal256 = true fixes every case
enable_decimal256 |
99 999 rows | 100 000 rows | 199 998 rows |
|---|---|---|---|
false (default) |
ok | one tenth | negative |
true |
ok | ok | ok |
What You Expected?
Either the correct total, or an error. check_overflow_for_decimal is 1 by
default and is documented as the switch that makes decimal overflow an error;
it should cover the aggregate the same way it covers a + a and a * 2.
Returning a positive, plausible number that is one tenth of the answer is the
one outcome that should not happen.
How to Reproduce?
CREATE DATABASE IF NOT EXISTS ovf;
-- 1. the contract, on two rows
CREATE TABLE ovf.two (id INT, a DECIMAL(38,0))
DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES('replication_num'='1');
INSERT INTO ovf.two VALUES (1, 99999999999999999999999999999999999999),
(2, 99999999999999999999999999999999999999);
SELECT a + a FROM ovf.two LIMIT 1; -- E-255 Arithmetic overflow (correct)
SELECT a * 2 FROM ovf.two LIMIT 1; -- E-255 Arithmetic overflow (correct)
SELECT SUM(a) FROM ovf.two; -- -14028236692093846346337460743176821145
-- <-- no error
-- 2. it is persisted by an AGGREGATE KEY table
CREATE TABLE ovf.agg (k INT, a DECIMAL(38,0) SUM)
AGGREGATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES('replication_num'='1');
INSERT INTO ovf.agg VALUES (1, 99999999999999999999999999999999999999);
INSERT INTO ovf.agg VALUES (1, 99999999999999999999999999999999999999);
SELECT a FROM ovf.agg; -- -14028236692093846346337460743176821145
-- 3. the two regimes, with an ordinary 34-digit value
CREATE TABLE ovf.deep (id BIGINT, v DECIMAL(38,0))
DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES('replication_num'='1');
-- load 320000 rows of 10^33 (the script below does this by doubling)
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 99999) x; -- correct
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 100000) x; -- one tenth
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 170142) x; -- negative
SET enable_decimal256 = true;
SELECT SUM(v) FROM (SELECT v FROM ovf.deep LIMIT 100000) x; -- correctA self-contained script that builds the data and prints the tables above:
python3 repro.py --port 9030 # parts 1 and 2 of the report
python3 verify-main.py --port 9030 # regime boundaries, all aggregate paths, AGGREGATE KEYAnything Else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Source: apache/doris