get_historical_features raises TypeError on a zero-row entity_df (Dask/file offline store)
Expected Behavior
get_historical_features with an entity_df that happens to have zero rows should
return an empty result with the right columns. A zero-row entity frame is a normal
degenerate case in batch scoring — the upstream query simply matched nothing that run.
Current Behavior
It raises an opaque pandas error from deep inside the Dask offline store:
TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArrayTraceback tail:
File "sdk/python/feast/infra/offline_stores/offline_store.py", line 178, in to_arrow
features_table = self._to_arrow_internal(timeout=timeout)
File "sdk/python/feast/infra/offline_stores/dask.py", line 103, in _to_arrow_internal
df = self.evaluation_function().compute()
File "sdk/python/feast/infra/offline_stores/dask.py", line 317, in evaluate_historical_retrieval
df_to_join = _filter_ttl(
File "sdk/python/feast/infra/offline_stores/dask.py", line 1206, in _filter_ttl
df_to_join = df_to_join.persist()Steps to reproduce
Identical construction, only the row count differs, against a file offline store:
def mk(n):
return pd.DataFrame({
"driver_id": [1001] * n,
"event_timestamp": pd.to_datetime([datetime(2026, 2, 1)] * n, utc=True),
})
fs.get_historical_features(entity_df=mk(1), features=["driver_stats:conv_rate"]).to_df()
# OK, shape=(1, 3)
fs.get_historical_features(entity_df=mk(0), features=["driver_stats:conv_rate"]).to_df()
# TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArraySpecifications
- Version:
master@5ad5592390febfca60c9d88edf7daccbdd156fd6 - Platform: Linux x86_64, Python 3.11.15, dask 2026.8.0
- Subsystem: offline store (Dask / file)
Possible Solution
_normalize_timestamp (sdk/python/feast/infra/offline_stores/dask.py:1139) makes
timestamp columns tz-aware with a row-wise apply:
df_to_join[timestamp_field] = df_to_join[timestamp_field].apply(
lambda x: x if x.tzinfo else x.replace(tzinfo=timezone.utc),
meta=(timestamp_field, "datetime64[ns, UTC]"),
)meta declares a tz-aware result, but with zero rows the lambda never runs, so the
computed partition stays datetime64[ns]. Declared and actual dtypes then diverge, and
the tz-naive vs tz-aware comparison in _filter_ttl
(sdk/python/feast/infra/offline_stores/dask.py:1183) raises.
Replacing the row-wise apply with a vectorized, empty-safe conversion fixes it and is
faster on non-empty frames too: pick dt.tz_localize("UTC") for a tz-naive column and
dt.tz_convert("UTC") otherwise, based on the column's declared dtype. That yields the
correct dtype even when the partition is empty.
A regression test covering a zero-row entity_df through
get_historical_features(...).to_df() on the Dask store would pin this down; the
current suite only exercises non-empty entity frames, which is why CI stays green.
Happy to send a PR for this.
Source: feast-dev/feast