_prepare_returns cache key collision changes Series/DataFrame type and column metadata
_prepare_returns cache key collision changes Series/DataFrame type and column metadata
Description
utils._prepare_returns() can return a different pandas container type from the input because its cache key does not distinguish a Series from an equivalent single-column DataFrame.
The result depends on call order:
- If the DataFrame is prepared first, preparing the equivalent Series returns a DataFrame.
- If the Series is prepared first, preparing the equivalent DataFrame returns a Series.
DataFrame column labels are also missing from the key, so two DataFrames with identical values but different column names share a cache entry and the second result receives the first DataFrame's columns.
This appears to be a possible root cause of the downstream HTML report error reported in #383:
Index(...) must be called with a collection of some kind, 'Strategy' was passed
Minimal reproduction
import pandas as pd
import quantstats as qs
index = pd.date_range("2024-01-01", periods=3)
series = pd.Series([0.01, -0.02, 0.03], index=index, name="Strategy")
frame = series.to_frame()
qs.utils._PREPARE_RETURNS_CACHE.clear()
prepared_frame = qs.utils._prepare_returns(frame)
prepared_series = qs.utils._prepare_returns(series)
print(type(prepared_frame).__name__)
print(type(prepared_series).__name__)
print(
qs.utils._generate_cache_key(frame, 0.0, None)
== qs.utils._generate_cache_key(series, 0.0, None)
)Actual output
DataFrame
DataFrame
TrueReversing the two _prepare_returns() calls produces:
Series
SeriesColumn metadata is affected as well:
frame_a = pd.DataFrame({"A": [0.01, -0.02, 0.03]}, index=index)
frame_b = pd.DataFrame({"B": [0.01, -0.02, 0.03]}, index=index)
qs.utils._PREPARE_RETURNS_CACHE.clear()
qs.utils._prepare_returns(frame_a)
print(qs.utils._prepare_returns(frame_b).columns.tolist())Actual output:
['A']Expected output:
['B']Expected behavior
_prepare_returns() should preserve the input container type and metadata independently of earlier calls:
- Series input returns a Series.
- DataFrame input returns a DataFrame.
- DataFrame column labels are preserved.
Root cause
_generate_cache_key() uses the same expression for Series and DataFrame inputs:
data_hash = pd.util.hash_pandas_object(data).sum()For a Series and an equivalent single-column DataFrame, this produces the same value. The generated key contains only data_hash, rf, and nperiods; it omits the pandas container type and column/name metadata.
Suggested fix
Include the following in the cache key:
- Container type.
- Series name for Series inputs.
- Column labels for DataFrame inputs.
- Potentially dtype metadata if it affects the prepared result.
Regression tests should call _prepare_returns() in both orders and verify that type and column/name metadata are preserved.
Environment
- QuantStats: 0.0.81
- pandas: 2.3.3
- Python: 3.12.12
Source: ranaroussi/quantstats