#6101·paradedb

BM25 indexes reject valid NUMERIC typmods with |scale| > 18

Author: mithuncyCreated Aug 26, 2026Updated Sep 17, 2026
Labelsbugpriority-medium

What happens?

Valid PostgreSQL NUMERIC(p, s) columns fail to build a BM25 index when p <= 18 and |s| > 18.

ParadeDB selects Numeric64 using only the precision. That creates an I64-backed field with scale 20, but Decimal64NoScale supports only scales from -18 through 18, so indexing rejects a type that PostgreSQL accepts.

To Reproduce

sql
CREATE EXTENSION IF NOT EXISTS pg_search;

CREATE TABLE numeric_scale_repro (
    id int PRIMARY KEY,
    value numeric(3,20)
);

INSERT INTO numeric_scale_repro
VALUES (1, 0.00000000000000000123);

CREATE INDEX numeric_scale_repro_idx
ON numeric_scale_repro USING bm25 (id, value)
WITH (key_field = 'id');

The CREATE INDEX fails with:

ERROR: could not parse field `value`: NUMERIC conversion error: Failed to convert NUMERIC '0.00000000000000000123' to I64 with scale 20: Failed to scale '0.00000000000000000123' with scale 20: InvalidFormat("Scale 20 exceeds maximum 18 for Decimal64NoScale"). This may occur if the value exceeds i64 range after scaling.

The failure is symmetric for negative scales: numeric(3,-20) also fails. The boundary is exactly 18: numeric(3,18) succeeds, while numeric(3,19) fails.

The empty-index lifecycle fails later through the same conversion:

sql
CREATE TABLE numeric_scale_delayed (
    id int PRIMARY KEY,
    value numeric(3,20)
);

CREATE INDEX numeric_scale_delayed_idx
ON numeric_scale_delayed USING bm25 (id, value)
WITH (key_field = 'id');

INSERT INTO numeric_scale_delayed
VALUES (1, 0.00000000000000000123);

SELECT id
FROM numeric_scale_delayed
WHERE id @@@ pdb.all();

Here, CREATE INDEX and INSERT succeed, but the first search fails with the same conversion error when the mutable segment is materialized.

Expected behavior

Both index-build and mutable-segment paths should accept every PostgreSQL NUMERIC typmod supported by the selected ParadeDB representation. The final query above should return ID 1.

The representation-selection invariant should include the encoder's scale limit: choose Numeric64 only when both precision <= 18 and |scale| <= 18; otherwise use a representation that supports the typmod (currently NumericBytes), unless Decimal64NoScale is extended to support PostgreSQL's wider scale range.

Suspected cause

SearchFieldType::try_from_type_info checks the precision limit but not the scale limit before selecting Numeric64.

Regression coverage should include numeric(3,18), numeric(3,19), and numeric(3,-19), plus both populated CREATE INDEX and empty-index → INSERT → query lifecycles.

ParadeDB Version

0.25.4 (standalone pg_search, PostgreSQL 17, macOS arm64)