FileIO.load(file_directories=...) reuses first file's inferred format for all files
Author: chirag127Created Jul 4, 2026Updated Jul 4, 2026
Problem
When FileIO.load() is called with file_directories and format=None, the format inferred from the first file is reused for every subsequent file, corrupting reads on mixed-format directories.
File
mage_ai/io/file.py lines 45-49:
for fp in file_paths:
if format is None:
format = self._get_file_format(fp)
df = self._read(fp, format, **kwargs)After iter 1, format is set, so if format is None is False for iter 2+.
Steps
- Create dir with
a.csvandb.parquet. FileIO().load(file_directories=['dir'])
Expected
Each file read with its own inferred format.
Actual
b.parquet read as CSV; pandas raises UnicodeDecodeError or ParserError.
Env
mage-ai 0.9.79, Python >=3.10 (per setup.py:21,36).
Fix
Move format = None reset inside the loop, or use a local var per iteration.
Thanks for maintaining mage-ai/mage-ai!
Source: mage-ai/mage-ai