Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#4107·hypothesis

Support `async` methods in stateful testing

Author: Zac-HDCreated Sep 18, 2024Updated Nov 5, 2025
Labelsnew-featureinterop

@given() doesn't natively support async functions, but thanks to https://github.com/HypothesisWorks/hypothesis/pull/1343 that can be handled downstream e.g. in pytest-trio. However, we don't have a good solution for stateful testing with async methods, and it's harder to add downstream - for example, hypothesis-trio copy-pasted all of stateful.py to add a few awaits, but is now several years of bugfixes and performance improvements behind.

I'd like to allow RuleBasedStateMachines to define a mix of sync and async methods for rules, invariants, and teardown.

Instead of a firm design, some notes on options:

  • We're not aiming to run multiple rules concurrently, just support concurrency within each rule. This might mean we can [asyncio/anyio/trio].run() each async method separately, which greatly simplifies the implementation.

    • Add a new method __execute_fn(self, fn, *args, /, **kwargs); this can default to return fn(*args, **kwargs) with the obvious more-complicated options to support async frameworks (branching on inspect.iscoroutinefunction(fn)). We can ship first-party subclasses {AnyIO,Asyncio,Trio}RuleBasedStateMachine with the appropriate implementations.
  • On the other hand, if we need to maintain some (async) state over the whole run, that gets more complicated.

    • ⭐ we'll want to add a new interface to hold context managers open for the duration of the test - perhaps teach @initialize() to recognize methods wrapped with @contextlib.(async)contextmanager and handle them appropriately?
      • this sounds pretty nice anyway, which hints that we're in this scenario
    • we'll have to make the whole implementation syntactically async, and then have some reasonable logic for the all-synchronous case to keep our current UX (traceback trimming plus a thin asyncio impl?). I still want to avoid taking a position on what async framework users should use, and think I still prefer framework-specific subclasses defining an executor method over a stateful_async_backend setting.
    • I don't want to deal with unstructured concurrency from e.g. asyncio launching a background task - when await whatever_rule() returns, it should be finished. No idea what we could do about that though, beyond documenting that it's not supported.
      • Trio can have lesser issues from e.g. a spanning context with some background task running; the trio.testing.wait_all_* functions might be useful to let things settle down between rules.

Thoughts?

Source: HypothesisWorks/hypothesis

View original on GitHubView discussion on GitHub