#65237·pandas

BUG: read_csv() into FloatingArrays converts np.nan into pd.NA, even with distinguish_nan_and_na=True

Author: carlocastoldiCreated Apr 14, 2026Updated Sep 28, 2026
LabelsBugIO CSVNA - MaskedArrays

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

python
import io
import pandas as pd
import numpy as np

csv = """,x
a,1.0
b,
c,nan"""

with pd.option_context("future.distinguish_nan_and_na", True):
    s = pd.read_csv(io.StringIO(csv), index_col=0, dtype={0: str, 1: pd.Float64Dtype()},
                     keep_default_na=False, na_values=[""])["x"]
s

Issue Description

Working with data having both pd.NA and np.nan is possible and, at least until pandas2, it was possible to save such dataframes as CSV in a format that keeps a distinction between the two (see #65227).

However, in the example above pandas fails with:

ValueError: Unable to parse string "nan" at position 2

The current workaround consists in reading the CSV as string, and manually map its value to floats:

python
import io
import pandas as pd
import numpy as np

csv = """,x
a,1.0
b,
c,nan"""


def convert_nan(val: str):
    match val:
        case "":
            return pd.NA
        case _:
            return np.float64(val) # includes np.nan

with pd.option_context("future.distinguish_nan_and_na", True):
    s1 = pd.read_csv(io.StringIO(csv), index_col=0, dtype=str, keep_default_na=False)["x"]
    s2 = s1.map(convert_nan)
    s3 = s2.convert_dtypes()
s1,s2,s3

returns:

(a    1.0
 b       
 c    nan
 Name: x, dtype: str,
 a     1.0
 b    <NA>
 c     NaN
 Name: x, dtype: object,
 a     1.0
 b    <NA>
 c     NaN
 Name: x, dtype: Float64)

Expected Behavior

I would expect read_csv() to distinguish between NA and NaN in the columns with dtype=Float64. Even more so if distinguish_nan_and_na=True.

 a     1.0
 b    <NA>
 c     NaN
 Name: x, dtype: Float64

Installed Versions

INSTALLED VERSIONS

commit : ab90747e3dae0e69b1bdbf083820b8075689b34b python : 3.13.13 python-bits : 64 OS : Linux OS-release : 6.17.0-20-generic Version : #20~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Mar 19 01:28:37 UTC 2 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_GB.UTF-8 LOCALE : en_GB.UTF-8

pandas : 3.0.2 numpy : 2.4.4 dateutil : 2.9.0.post0 pip : 26.0.1 Cython : None sphinx : None IPython : 9.5.0 adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.13.5 bottleneck : None fastparquet : None fsspec : 2025.7.0 html5lib : None hypothesis : None gcsfs : None jinja2 : 3.1.6 lxml.etree : None matplotlib : 3.10.6 numba : 0.65.0 numexpr : 2.11.0 odfpy : None openpyxl : None psycopg2 : None pymysql : None pyarrow : 21.0.0 pyiceberg : None pyreadstat : None pytest : 8.4.1 python-calamine : None pytz : None pyxlsb : None s3fs : None scipy : 1.16.1 sqlalchemy : None tables : 3.10.2 tabulate : None xarray : None xlrd : None xlsxwriter : None zstandard : None qtpy : None pyqt5 : None