Numeric64 pushdown rounds query literals and changes comparison semantics
What happens?
Predicate pushdown on Numeric64 fields rounds query literals to the column's declared scale before constructing the Tantivy query.
PostgreSQL does not apply the column typmod to a comparison operand. For a numeric(10,2) column, PostgreSQL compares against the exact value 12.345. ParadeDB instead converts the operand as if it were being inserted into the column:
12.345 → 12.35 → Numeric64 value 1235This produces incorrect equality and range results.
This is distinct from #6101: the column below has a supported scale of 2, and the index builds successfully. The failure occurs while translating query literals.
To Reproduce
CREATE EXTENSION IF NOT EXISTS pg_search;
CREATE TABLE numeric_literal_repro (
id int PRIMARY KEY,
price numeric(10,2)
);
INSERT INTO numeric_literal_repro
VALUES (1, 12.34), (2, 12.35);
CREATE INDEX numeric_literal_repro_idx
ON numeric_literal_repro USING bm25 (id, price)
WITH (key_field = 'id');First, obtain PostgreSQL's native results:
SET paradedb.enable_custom_scan = off;
SELECT id
FROM numeric_literal_repro
WHERE price = 12.345
ORDER BY id;
SELECT id
FROM numeric_literal_repro
WHERE price <= 12.345
ORDER BY id;Native results:
price = 12.345: no rows
price <= 12.345: {1}Now execute the predicates through the v2 pushdown path:
SET paradedb.enable_custom_scan = on;
SELECT id
FROM numeric_literal_repro
WHERE id @@@ pdb.all()
AND price = 12.345
ORDER BY id;
SELECT id
FROM numeric_literal_repro
WHERE id @@@ pdb.all()
AND price <= 12.345
ORDER BY id;
RESET paradedb.enable_custom_scan;Observed pushed-down results:
price = 12.345: {2}
price <= 12.345: {1,2}The full operator comparison is:
| Predicate | PostgreSQL | Pushed down |
|---|---|---|
= 12.345 |
{} |
{2} |
<> 12.345 |
{1,2} |
{1} |
< 12.345 |
{1} |
{1} |
<= 12.345 |
{1} |
{1,2} |
> 12.345 |
{2} |
{} |
>= 12.345 |
{2} |
{2} |
Expected behavior
Pushed-down comparisons must return the same rows as PostgreSQL when the query operand contains more fractional digits than the indexed column's scale.
For a scale-2 field, 12.345 lies between the physical Numeric64 values representing 12.34 and 12.35. Therefore:
= 12.345should match nothing.<> 12.345should match every non-null value.- Upper bounds should use the grid value below the literal.
- Lower bounds should use the grid value above the literal.
- Exact grid values should preserve their inclusive or exclusive bound.
Suspected cause
The query path eventually calls scale_owned_value() / scale_i64(), which uses Decimal64NoScale::new(). That constructor rounds to the requested scale, which is appropriate for encoding stored column values but not for translating comparison operands.
Stored-value conversion and query-literal conversion should be handled separately so pushed-down predicates preserve PostgreSQL comparison semantics.
Regression coverage should include all six comparison operators with:
- On-grid and off-grid literals
- Positive and negative values
- Positive, zero, and negative column scales
ParadeDB Version
0.25.4 (standalone pg_search, PostgreSQL 17, macOS arm64)
Source: paradedb/paradedb