contrib.concurrent.process_map raises ValueError with unsized iterables and no explicit chunksize

• I have marked all applicable categories:
  • exception-raising bug
  • visual output bug
• I have visited the source website, and in particular read the known issues
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:
  python复制

  import tqdm, sys
  print(tqdm.__version__, sys.version, sys.platform)

4.70.0 3.13.3 (main, Jun 3 2025, 19:17:36) [GCC 11.4.0] linux

tl;dr:

Passing only generators of unknown size to process_map() raises a ValueError in the latest stable release of tqdm==4.70.0:

File "lib/python3.13/site-packages/tqdm/contrib/concurrent.py", line 104, in _min_map_len
    return min(n for it in iterables if (n := length_hint(it, -1)) >= 0)
ValueError: min() iterable argument is empty

Min. Reproducible Example

Python 3.13.3 (main, Jun  3 2025, 19:17:36) [GCC 11.4.0] on linux
>>> from tqdm.contrib.concurrent import process_map
>>> process_map(print, (i for i in range(5)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    import platform
  File ".venv/lib/python3.13/site-packages/tqdm/contrib/concurrent.py", line 252, in process_map
    shortest_iterable_len = _min_map_len(iterables)
  File ".venv/lib/python3.13/site-packages/tqdm/contrib/concurrent.py", line 104, in _min_map_len
    return min(n for it in iterables if (n := length_hint(it, -1)) >= 0)
ValueError: min() iterable argument is empty

Note that length_hint(range(5), -1) returns 4, but length_hint((i for i in range(5), -1) returns -1 because length_hint assumes it knows nothing about the contents of the generator and returns the specified default of -1.

Explanation

tqdm.contrib.concurrent.process_map(fn, *iterables, chunksize=None) attempts to guess the size of iterables and raise a warning if the shortest iterable is longer than 1000. It uses builtin operator.length_hint to do the size guessing.

Some iterables have no size information, such as generators.

History

In 4.69.1, the condition for raising the warning was max(map(length_hint, iterables)) > 1000. This technically did the wrong thing by checking for the longest iterator (the shortest is what matters for the warning), but it did not raise errors when passed an unsized iterator because *unsized iterators would become a 0 element in the 'list' passed to max(), so if iterables was not empty, then max's arg was not empty.

In #1473, the logic was fixed to use the shortest iterable. However, because length_hint(iterable) returns 0 for unsized iterable, a trivial change such as min(map(length_hint, iterables)) would've just always returned 0 if there were any unsized iterators. SO, they changed the code to REMOVE any unsized iterators and therefore ignore them in the size calculations:

min(n for it in iterables if (n := length_hint(it, -1)) >= 0).

Unsized iterators passed to length_hint(it, -1) become -1, which is less than 0, so min() never sees it. THEREFORE, if ALL iterators passed to process_map are unsized, then min() would be empty, since they get filtered out for being unsized. At last, we have ValueError: min() iterable argument is empty.

The logic from #1473 was released in 4.70.0, where the error now occurs.

Fix

There is currently an open PR #1788 which I presume fixes the issue, by simply providing a default to min() for when it's passed something empty.

postscript

i assume nobody's gonna read this issue, especially since there's already a PR for it. but if you're reading this then thanks for taking the time i guess? i only typed all this out to link all the old PRs and commits with the new PR. take care :)