1.3.0 regression: false errors in max/min key lambdas when default= is passed
Describe the Bug
When max() or min() is called with both key= and default=, Pyrefly types the
key callable's parameter from the expected type of the whole call rather than from
the iterable's element type. In the common default=None case the lambda parameter
picks up None, and every attribute access in the lambda body fails.
from dataclasses import dataclass
@dataclass
class Item:
rank: int
def bug(items: list[Item]) -> Item | None:
return max(items, key=lambda i: i.rank, default=None)ERROR Object of class `NoneType` has no attribute `rank` [missing-attribute]
--> repro.py:10:37
|
10 | return max(items, key=lambda i: i.rank, default=None)
| ^^^^^^items is list[Item] and holds no None, so i should be Item.
This is a regression in 1.3.0 — 1.2.0 accepts the same file, as do pyright 1.1.414
and mypy 2.3.1 --strict.
Narrowing
The error requires an expected type, and that expected type has to be a union, though not every union produces it:
| Variant | Result |
|---|---|
def f(...) -> Item | None: return max(items, key=lambda i: i.rank, default=None) |
error |
best: Item | None = max(items, key=lambda i: i.rank, default=None) |
error |
best = max(items, key=lambda i: i.rank, default=None) (no expected type) |
clean |
def f(...) -> Item: return max(items, key=lambda i: i.rank, default=Item(0)) |
clean |
def f(...) -> Item: return max(items, key=lambda i: i.rank) (no default=) |
clean |
def f(...) -> object | None: return max(items, key=lambda i: i.rank, default=None) |
clean |
max(items, key=rank_of, default=None) with an annotated def rank_of(i: Item) -> int |
clean |
min() behaves identically. The bug is not specific to None: in a function
returning Item | str, max(items, key=lambda i: i.rank, default="sentinel")
reports Object of class `str` has no attribute `rank` .
Passing the call as an argument to a function whose parameter is annotated
Item | None also produces the error. Reversing the keyword arguments to
max(items, default=None, key=lambda i: i.rank) does not avoid it.
The overload selected is:
@overload
def max(iterable: Iterable[_T1], /, *, key: Callable[[_T1], SupportsRichComparison], default: _T2) -> _T1 | _T2: ...The error suggests that contextual checking uses None as a possible lambda
parameter type, even though the iterable contains only Item. It does not by
itself establish that the final inferred value of _T1 is Item | None.
The trigger is that max and min are overloaded. Two calls that differ only in
whether the callee is overloaded isolate it:
@overload
def pick[T1](it: Iterable[T1], /, *, key: Callable[[T1], int]) -> T1: ...
@overload
def pick[T1, T2](it: Iterable[T1], /, *, key: Callable[[T1], int], default: T2) -> T1 | T2: ...
def pick(it: Any, /, *, key: Any, default: Any = None) -> Any: ...
# Same signature as `pick`'s second overload, without `@overload`.
def pick_single[T1, T2](it: Iterable[T1], /, *, key: Callable[[T1], int], default: T2) -> T1 | T2: ...
def f(items: list[Item]) -> Item | None:
return pick(items, key=lambda i: i.rank, default=None) # error
return pick_single(items, key=lambda i: i.rank, default=None) # cleanAn overloaded function whose second overload takes default: None and returns
T1 | None reports the same error, so two distinct return type variables are
neither necessary nor sufficient to trigger it.
The union return type matters for a different reason: _T1 | _T2 is assignable to
no single member of the expected union. Pyrefly tries each member of a union
expected type as a standalone return hint before falling back to the whole union, so
with default= every per-member attempt fails, and each failed attempt checks the
lambda body with that member as the parameter type. The errors from the discarded
attempts are kept. Three measurements fit that description:
- The error count tracks the width of the expected union.
Item | None | bytesreports two errors on the single lambda,Item | None | bytes | complexreports three, andItem | Nonewidened with 30 unrelated classes reports 31. - An expected union of more than 32 members is silent, matching
MAX_HINT_WIDTH, beyond which the individual members are not tried. - An expected union whose first member absorbs the whole return type is clean:
returning
object | Nonereports nothing.
The reported diagnostic is whatever the lambda body produces, so missing-attribute
is not the only error kind involved. key=lambda r: r[0] over a list[list[int]]
reports unsupported-operation, and calling an annotated function in the lambda
body reports bad-argument-type.
Full repro
# Save to repro.py
from collections.abc import Callable, Iterable
from dataclasses import dataclass
from typing import Any, overload
@dataclass
class Item:
rank: int
# --- Errors --------------------------------------------------------------
def bug(items: list[Item]) -> Item | None:
return max(items, key=lambda i: i.rank, default=None)
def bug_min(items: list[Item]) -> Item | None:
return min(items, key=lambda i: i.rank, default=None)
def bug_annotated_local(items: list[Item]) -> None:
best: Item | None = max(items, key=lambda i: i.rank, default=None)
print(best)
def bug_not_none(items: list[Item]) -> Item | str:
return max(items, key=lambda i: i.rank, default="sentinel")
def bug_other_error_kind(rows: list[list[int]]) -> list[int] | None:
return max(rows, key=lambda r: r[0], default=None)
@overload
def pick[T1](it: Iterable[T1], /, *, key: Callable[[T1], int]) -> T1: ...
@overload
def pick[T1, T2](
it: Iterable[T1], /, *, key: Callable[[T1], int], default: T2
) -> T1 | T2: ...
def pick(it: Any, /, *, key: Any, default: Any = None) -> Any: ...
def bug_without_stdlib(items: list[Item]) -> Item | None:
return pick(items, key=lambda i: i.rank, default=None)
# --- Clean ---------------------------------------------------------------
def ok_no_expected_type(items: list[Item]) -> None:
best = max(items, key=lambda i: i.rank, default=None)
print(best)
def ok_no_default(items: list[Item]) -> Item:
return max(items, key=lambda i: i.rank)
def ok_non_union_expected_type(items: list[Item]) -> Item:
return max(items, key=lambda i: i.rank, default=Item(0))
def ok_absorbing_union_member(items: list[Item]) -> object | None:
return max(items, key=lambda i: i.rank, default=None)
def ok_named_function(items: list[Item]) -> Item | None:
def rank_of(i: Item) -> int:
return i.rank
return max(items, key=rank_of, default=None)
# Same signature as `pick`'s second overload, without `@overload`.
def pick_single[T1, T2](
it: Iterable[T1], /, *, key: Callable[[T1], int], default: T2
) -> T1 | T2:
raise NotImplementedError
def ok_not_overloaded(items: list[Item]) -> Item | None:
return pick_single(items, key=lambda i: i.rank, default=None)uvx --from pyrefly==1.2.0 pyrefly check --preset default repro.py
uvx --from pyrefly==1.3.0 pyrefly check --preset default repro.pyPyrefly 1.3.0 reports six errors, all in the first group:
ERROR Object of class `NoneType` has no attribute `rank` [missing-attribute]
--> repro.py:14:37
ERROR Object of class `NoneType` has no attribute `rank` [missing-attribute]
--> repro.py:18:37
ERROR Object of class `NoneType` has no attribute `rank` [missing-attribute]
--> repro.py:22:50
ERROR Object of class `str` has no attribute `rank` [missing-attribute]
--> repro.py:27:37
ERROR `None` is not subscriptable [unsupported-operation]
--> repro.py:31:36
ERROR Object of class `NoneType` has no attribute `rank` [missing-attribute]
--> repro.py:44:38
INFO 6 errorsPyrefly 1.2.0, pyright 1.1.414, and mypy 2.3.1 --strict all report nothing.
The introduction is between Pyrefly 1.3.0.dev2 and 1.3.0.dev3: dev2 reports no
errors, while dev3 reports the same six errors as 1.3.0.
uvx --from pyrefly==1.3.0.dev2 pyrefly check --preset default repro.py
uvx --from pyrefly==1.3.0.dev3 pyrefly check --preset default repro.pyWithin that range the introducing commit is fdb78cf79, "fix Infer input type of
lambda based on 2nd argument to map() #438" (PR #4215). Built from source, its parent
31c0afb96 reports none of these errors and fdb78cf79 reports all of them. That
change keeps lambdas unevaluated through overload preparation so that they are
contextually typed against the selected overload, instead of being inferred once
without context.
The errors are still present on main at ef1bec60b.
Sandbox Link
(Only applicable for extension issues) IDE Information
No response
Source: facebook/pyrefly