SQL: an unaliased constant in the SELECT list of a `GROUP BY` query raises `unable to find column "literal"`
Checks
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of Polars.
Reproducible example
import polars as pl
t = pl.DataFrame({"a": [0, 1, 2, 3, 4]})
ctx = pl.SQLContext(t=t.lazy())
# Valid SQL: a constant projected over 5 groups -> 5 rows, all 2.
ctx.execute("SELECT 2 FROM t GROUP BY a").collect()
# polars.exceptions.ColumnNotFoundError: unable to find column "literal"; valid columns: ["a"]
# One alias, and the same query is fine:
ctx.execute("SELECT 2 AS two FROM t GROUP BY a").collect()
# shape: (5, 1) -- column "two", five rows of 2
# The DataFrame API builds exactly what the SQL should have built:
t.lazy().group_by("a").agg(pl.lit(2)).collect()
# shape: (5, 2) -- columns ["a", "literal"]
# And `GROUP BY ALL` takes a different path that works, naming the column "literal":
ctx.execute("SELECT 2 FROM t GROUP BY ALL").collect()
# shape: (5, 1) -- column "literal", five rows of 2Log output
polars.exceptions.ColumnNotFoundError: unable to find column "literal"; valid columns: ["a"]
Resolved plan until failure:
---> FAILED HERE RESOLVING 'select' <---
AGGREGATE[maintain_order: false]
[] BY [col("a")]
FROM
DF ["a"]; PROJECT */1 COLUMNS
This error occurred with the following context stack:
[1] 'select'
[2] 'select'
The aggregate list is empty — the constant was dropped on the way into the `GROUP BY` —
while the projection that follows still asks for the column it would have produced.Issue description
SELECT <constant> FROM t GROUP BY <key> is valid SQL: the constant is projected once per
group. DuckDB and SQLite both return one row per group for every query below. Polars
raises unable to find column "literal" at collect() time (the LazyFrame builds
fine), on both engine="in-memory" and engine="streaming".
It is not specific to any one kind of constant — every unaliased constant projection in a
GROUP BY query fails, whatever it is made of:
| query | polars | DuckDB / SQLite |
|---|---|---|
SELECT 2 FROM t GROUP BY a |
raises | 5 rows |
SELECT 1 + 1 FROM t GROUP BY a |
raises | 5 rows |
SELECT TRUE FROM t GROUP BY a |
raises | 5 rows |
SELECT 'x' FROM t GROUP BY a |
raises | 5 rows |
SELECT abs(-42) FROM t GROUP BY a |
raises | 5 rows |
SELECT COALESCE(42, 1) FROM t GROUP BY a |
raises | 5 rows |
SELECT 42 BETWEEN 1 AND 2 FROM t GROUP BY a |
raises | 5 rows |
SELECT 42 IN (9) FROM t GROUP BY a |
raises | 5 rows |
SELECT a, 2 FROM t GROUP BY a |
raises | 5 rows |
SELECT count(*), 2 FROM t GROUP BY a |
raises | 5 rows |
SELECT 2 FROM t GROUP BY a HAVING count(*) > 0 |
raises | 5 rows |
Change one detail and it works, which is what makes this look like an oversight rather than an unsupported feature:
| variant | result |
|---|---|
SELECT 2 AS two FROM t GROUP BY a |
5 rows — an alias is enough |
SELECT 2 FROM t GROUP BY ALL |
5 rows, column named literal |
SELECT DISTINCT 2 FROM t |
1 row |
SELECT 2 FROM t (no GROUP BY) |
5 rows |
t.lazy().group_by("a").agg(pl.lit(2)) |
5 rows — the DataFrame API is fine |
So the value, the column name (literal) and the grouping all work; it is only the SQL
GROUP BY path with an unaliased constant that breaks. GROUP BY ALL even produces the
exact frame the failing query should produce, under the exact column name the error says
it cannot find.
Where it comes from
crates/polars-sql/src/context.rs, process_group_by. Each SELECT-list expression is
classified into one of three buckets:
for (e, group_key) in projections.iter().zip(&projection_group_key) {
let matches_group_key = group_key.is_some();
let is_non_group_key_expr =
!matches_group_key && requires_group_processing(e, &group_by_keys_schema);
...
if is_non_group_key_expr {
...
splitter.aggregates.push(e); // goes into the aggregation
} else if !matches_group_key {
// Non-aggregated columns must be part of the GROUP BY clause
if let Expr::Column(_) | Expr::Function { ... FieldByName(_) ... } = e_inner {
if !group_by_keys_schema.contains(&field.name) {
polars_bail!(SQLSyntax: "'{}' should participate in the GROUP BY clause or an aggregate function", &field.name);
}
}
// <-- a constant is neither a group key nor an aggregate, is not a Column,
// so it falls out here: not pushed anywhere, silently dropped
}
}A constant needs no group processing, so is_non_group_key_expr is false; it does not
match a group key; and it is not an Expr::Column, so the "should participate in the
GROUP BY clause" bail does not fire either. It simply falls through the else if with
nothing done — never added to splitter.aggregates, never added to
post_agg_projection. The final projection then selects by output name and there is no
literal column to find.
The aliased case escapes because it is picked up earlier by the Expr::Alias arm
(projection_aliases), which is why one alias is the whole difference.
Expected behavior
SELECT 2 FROM t GROUP BY a returns one row per group, each holding 2, with the output
name Polars already uses for an unaliased literal — exactly what
SELECT 2 FROM t GROUP BY ALL and t.lazy().group_by("a").agg(pl.lit(2)) return today:
shape: (5, 1)
┌─────────┐
│ literal │
│ --- │
│ i32 │
╞═════════╡
│ 2 │
│ 2 │
│ 2 │
│ 2 │
│ 2 │
└─────────┘Installed versions
polars 1.44.2 (PyPI, latest release) -- reproduces
polars 2.0.0rc1 (PyPI) -- reproduces
polars main @ 3c771bd, built from source with `--profile fast-release` -- reproduces
Python 3.10, Ubuntu 22.04.2 LTS, x86_64Source: pola-rs/polars