Silent data loss when parsing an iterator into a union of sequences

Author: jcristCreated Feb 1, 2024Updated Sep 16, 2026

Labelsbug V2

Initial Checks

I confirm that I'm using Pydantic V2

Description

Given a field that's a union of sequences (where pydantic tries validating against each sequence type in turn), if an iterator is passed in that fails to match the first type in the union, pydantic will silently drop all data previously seen in the iterator as it continues to try to match against the next type.

I've checked and this bug is new in Pydantic V2, the following code passes fine on Pydantic V1

Example Code

python

from pydantic import BaseModel
from typing import Union


class Test(BaseModel):
    # ints first
    x: Union[list[int], list[str], None] = None
    # strs first
    y: Union[list[str], list[int], None] = None


def gen():
    for val in range(3):
        yield val


# Passes, since generator is only iterated over once
assert Test(x=gen()) == Test(x=[0, 1, 2])

# Fails, since the generator is iterated over twice
assert Test(y=gen()) == Test(y=[0, 1, 2])

Python, Pydantic & OS Version

pydantic version: 2.6.0
        pydantic-core version: 2.16.1
          pydantic-core build: profile=release pgo=true
                 install path: /home/jcristharif/miniconda3/envs/msgspec/lib/python3.9/site-packages/pydantic
               python version: 3.9.18 | packaged by conda-forge | (main, Dec 23 2023, 16:33:10)  [GCC 12.3.0]
                     platform: Linux-5.15.0-92-generic-x86_64-with-glibc2.35
             related packages: pyright-1.1.344 typing_extensions-4.9.0 mypy-1.8.0
                       commit: unknown

Source: pydantic/pydantic

View original on GitHub View discussion on GitHub