Polars hits the sweet spot between Pandas’ ease and Spark’s scale.
If you’ve ever waited on a groupby or cursed a memory error, this cheat sheet is for you.
I’ve pulled the patterns that save time in real pipelines, not just toy examples.
Bookmark this before your next ETL run.
Setup and Basics First, get Polars and a dataset.
The lazy API is the default now, so you’ll rarely need to call .lazy() explicitly.
Start with a CSV or Parquet file, or create a DataFrame from scratch. pip install polars pyarrow import polars as pl df = pl.read_csv('data.csv') # or pl.read_parquet() df = pl.DataFrame({'a': [1, 2], 'b': ['x', 'y']}) Selecting and Filtering Polars uses expressions, not strings.
This feels odd at first but pays off when you chain operations.
The syntax is consistent: every column is an expression you can transform, filter, or aggregate. df.select(['a', 'b']) # columns by name df.select(pl.col('a').alias('renamed')) df.filter(pl.col('a') > 10) df.filter(pl.col('b').is_in(['x', 'z'])) df.filter(pl.col('a').is_null()) Transforming Data Polars expressions are composable.
You can nest them, reuse them, and even store them in variables.
This is where the library shines over Pandas. df.with_columns(pl.col('a').cast(pl.Float64)) df.with_columns(pl.col('a').fill_null(0)) df.with_columns((pl.col('a') * 2).alias('a_doubled')) df.with_columns(pl.col('b').str.to_uppercase()) df.with_columns(pl.col('a').is_between(10, 20)) Grouping and Aggregations Groupbys in Polars are lazy by default.
This means you can stack multiple aggregations without materializing intermediate results.
The syntax is clean, but watch out for the order of operations. df.group_by('b').agg(pl.col('a').sum()) df.group_by('b').agg([pl.col('a').mean(), pl.col('a').max()]) df.group_by('b').agg(pl.col('a').quantile(0.9)) df.group_by_dynamic('timestamp', every='1d').agg(pl.col('a').sum()) Joins and Concatenation Joins in Polars are explicit.
You’ll specify the join type and the columns to join on.
Concatenation is straightforward, but remember that Polars is strict about schema matching. df.join(other, on='key', how='inner') df.join(other, on='key', how='left') df.hstack([other]) # column-wise df.vstack([other]) # row-wise pl.concat([df1, df2], how='diagonal') # schema-safe union Performance Tips Polars is fast, but you can make it faster.
The lazy API is your friend.
Use it for any pipeline longer than a few operations.
Also, avoid Python loops.
Polars expressions are vectorized, so let the engine do the work.
Use .lazy() and .collect() for pipelines with multiple steps.
Prefer Parquet over CSV for I/O.
It’s faster and smaller.
Use .with_columns() instead of multiple .select() calls.
Avoid .apply() unless absolutely necessary.
Use expressions first.
Set pl.Config.set_fmt_str_lengths(100) to see full strings in debug output.
Debugging and Inspection Polars has great tools for debugging.
The .explain() method shows the query plan, which is invaluable for optimizing lazy pipelines.
For quick checks, use .head() or .sample(). df.head(5) # first 5 rows df.sample(5) # random 5 rows df.describe() # summary stats df.schema # column names and types lazy_df.explain() # query plan for lazy DataFrames Polars won’t replace Pandas for every task, but it’s the right tool for most data pipelines.
The syntax takes a day to learn and a week to master.
Once it clicks, you’ll write faster, cleaner code.
Keep this cheat sheet handy until the patterns stick.
This post was originally published on my site.
Read the full article and more →