#8596·datasets

load_dataset silently accepts out-of-range ClassLabel indices when the source dtype already matches

Author: caiotheodoroCreated Sep 10, 2026Updated Sep 16, 2026

Describe the bug

Loading a file whose parsed Arrow dtype already matches a declared ClassLabel's storage type (int64) skips the range check entirely. Out-of-range indices load with no error and no warning, and only surface later at int2str time or as silently wrong labels during training.

python
from datasets import Dataset, Features, ClassLabel
import json, tempfile, os

with tempfile.TemporaryDirectory() as d:
    p = os.path.join(d, "data.jsonl")
    with open(p, "w") as f:
        f.write(json.dumps({"label": 5}) + "\n")
        f.write(json.dumps({"label": 1}) + "\n")
    ds = Dataset.from_json(p, features=Features({"label": ClassLabel(names=["neg", "pos", "oth"])}))
    print(ds["label"])  # [5, 1], no error, only 3 classes declared

Same result with Parquet:

python
import pyarrow as pa, pyarrow.parquet as pq
from datasets import Dataset, Features, ClassLabel
import tempfile, os

with tempfile.TemporaryDirectory() as d:
    p = os.path.join(d, "data.parquet")
    pq.write_table(pa.table({"label": pa.array([5, 1], type=pa.int64())}), p)
    ds = Dataset.from_parquet(p, features=Features({"label": ClassLabel(names=["neg", "pos", "oth"])}))
    print(ds["label"])          # [5, 1]
    ds.features["label"].int2str(ds["label"][0])  # ValueError: Invalid integer class label 5, raised far from the load site

Dataset.from_dict with the same features= argument raises immediately instead:

python
Dataset.from_dict({"label": [5, 1]}, features=Features({"label": ClassLabel(names=["neg", "pos", "oth"])}))
# ValueError: Class label 5 greater than configured num_classes 3

So a .jsonl/.parquet file with a bad label and an in-memory dict with the same bad label behave differently: one raises at load time, the other loads silently and only fails (or worse, doesn't fail) downstream.

Cause

table_cast in src/datasets/table.py only runs the feature-level cast (cast_table_to_schema, which calls ClassLabel.cast_storage and does the range check) when the Arrow schema itself changes. pa.Schema.__eq__ ignores metadata, so when the file's native dtype (int64 for JSON/Parquet integers) already equals the ClassLabel's storage dtype, only the schema metadata differs and table_cast takes the cheap table.replace_schema_metadata(schema.metadata) branch, never calling cast_array_to_feature. from_dict/from_pylist don't go through table_cast for this — they encode through the feature directly, which is why they still validate.

This is the same root cause as #8494 (cast_column hitting the identical branch), which #8518 fixes — but #8518 scopes the fix to Dataset.cast/cast_column only (table_cast(..., validate_features=True)), and loaders keep calling table_cast with the default validate_features=False. That's a deliberate, discussed tradeoff (see the review thread on #8518: a full-validation table_cast call costs ~2.26ms vs ~0.006ms for the metadata-only path on a 50k-row/20-column batch, and every packaged loader's _cast_table sits on this path), not an oversight — but it leaves the load-time path, which is the more common way users attach a ClassLabel to raw integer data, silently unvalidated. I'm filing it separately from #8494 since that issue and #8518's fix are scoped to cast/cast_column and don't mention the loader path.

Confirmed CSV is not affected: CSV values parse as strings first, so the target int64 schema never equals the table's own schema and the full cast path always runs.

Expected behavior

Loading a file with features= containing a ClassLabel should raise the same ValueError as Dataset.from_dict when a label index is >= num_classes, or at least not silently return an inconsistent dataset.

Environment

datasets 5.0.2.dev0 installed from source at 4e56ddf20a63248a26e14ac848279246f5a35dac, Python 3.13.12, pyarrow 25.0.1, macOS arm64.