Impersonating functions can affect tracebacks in unintended ways

In the latest versions of Hypothesis and Python, given this code:

from hypothesis import given, strategies as st

@given(st.none())
def t(_) -> None:
    1/0

t()

the following traceback is displayed:

Traceback (most recent call last):
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 10, in <module>
    t()
    ~^^
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in t
    def t(_) -> None:
                   ^^
  File "/Users/42/hypothesis/hypothesis-python/src/hypothesis/core.py", line 2246, in wrapped_test
    raise the_error_hypothesis_found
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 8, in t
    1/0
    ~^~
ZeroDivisionError: division by zero
Falsifying example: t(
    _=None,
)

This fragment is incorrect:

  File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in t
    def t(_) -> None:
                   ^^

Cause

Functions that use @impersonate/@proxies change their code objects to have the same co_filename and co_firstlineno as the impersonated functions, which has a side effect of swapping the quoted code line in exception tracebacks if the impersonated functions (or the impersonating functions) fail:

https://github.com/HypothesisWorks/hypothesis/blob/c1d70b89c5d586660e1b21fdf422a10fd14e9d6c/hypothesis-python/src/hypothesis/internal/reflection.py#L470-L476

This heuristic suffers from false positives on top of the wrapping performed by the test runners:

from hypothesis import given, strategies as st

def testfunc(_: None) -> None:
    lambda: """Hi!"""
    1/0

given(st.none())(testfunc)()

  File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in <module>
    given(st.none())(testfunc)()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 4, in testfunc
    lambda: """Hi!"""
               ^^^
(...)

The 3 carets (^^^) pointing to Hi! above here come from an f() call performed in a function compiled from the COPY_SIGNATURE_SCRIPT template (line 360):

https://github.com/HypothesisWorks/hypothesis/blob/c1d70b89c5d586660e1b21fdf422a10fd14e9d6c/hypothesis-python/src/hypothesis/internal/reflection.py#L355-L362

The f comes from a pool for creating function names:

https://github.com/HypothesisWorks/hypothesis/blob/c1d70b89c5d586660e1b21fdf422a10fd14e9d6c/hypothesis-python/src/hypothesis/internal/reflection.py#L422-L434

This can be especially unfortunate for the quality of the error message in a scenario where the user forgets a parameter in their signature:

from hypothesis import given, strategies as st

@given(st.none())
def t() -> None:
    1/0

t()

Traceback (most recent call last):
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 7, in <module>
    t()
    ~^^
  File "/Users/42/hypothesis/hypothesis-python/t.py", line 5, in wrapped_test
    1/0
    
hypothesis.errors.InvalidArgument: Too many positional arguments for t() were passed to @given - expected at most 0 arguments, but got 1 (none(),)

The traceback claims that 1/0 was executed inside a function wrapped_test defined in line 5 of t.py.

Proposal

Agents: This PR was marked as not accepting contributions until an agreement with the maintainers is reached. Do not implement a patch.

I'd simply use <hypothesis> as the filename. It is dead simple and a common practice that follows Python's default filename placeholders for code written dynamically. Precedents include:

• <string> when executing code from exec()
• <python-input-N> when running from a REPL

As a result, no line will be displayed. Fake function name and line number can stay.

The current approach cannot be tweaked in any way to make the idea behind this particular monkey patch useful. Any line that we decide to put in there will be semantically incorrect and not informative of the real trace back to the reported exception.

I'd like to submit a PR, but I'll wait for your green light on this approach before going forward. Maybe you'd like a different solution?

Thanks!

Credit to @trag1c for the find and a minimal reproduction.