Add Polars as a native SQL cell engine?
Description
Related to #10396.
Any interest in adding Polars as an execution backend for SQL cells?
Polars has SQLContext, which can execute SQL against registered DataFrame and LazyFrame objects and return a LazyFrame:
ctx.execute(query, eager=False)
This could fit naturally into marimo's existing SQL engine abstraction as a PolarsEngine.
The main benefit would be keeping the query lazy and Polars-native end to end:
SQL cell -> Polars SQLContext -> LazyFrame
rather than executing the query in DuckDB and converting the result to Polars afterwards.
Are you willing to submit a PR? (You must receive approval from the team before submitting a PR.)
- Yes
Alternatives
No response
Additional context
At $day_job this would be useful to enable SQL for our less pythonic users, while still getting distributed query execution through polars cloud rather than client side as with duckdb. But I think it might have other applications. Polars and duckdb aren't completely overlapping in their feature sets
Source: marimo-team/marimo