#489·faust

Accessing windowed table outside stream

Author: orinciogCreated Dec 10, 2019Updated Jan 8, 2025

Hello!

I need to access a windowed table from outside stream. (The reason is that I need to obtain Top K users in a time interval. For this I store the count for each user in a tumbling table - key is the user and the value is the count of that user).

Using the following code,

bash
table_users = app.Table('users_tumbling', default=int,partitions=1) \
    .tumbling(timedelta(hours=1),key_index=True,expires=timedelta(hours=10)) \
    .relative_to_now()
@app.timer(5.0,on_leader=True)
async def publish_users_tumbling():
    logger.info(f"TIMER: {list(table_users.items())}" )

I get a list of users with 0 count: TIMER: [('Dolorem', 0), ('Velit', 0), ('Numquam', 0)]

  • I use relative_to_now() because with the default option, I got a RuntimeError: Operation outside of stream iteration

Using a simple table,

table_all=app.Table('users_all',default=int,partitions=1)
@app.timer(5.0,on_leader=True)
async def publish_users_all():
    logger.info(f"TIMER: {list(table_all.items())}" )

I get all counts: TIMER: [('Dolorem', 95), ('Velit', 32), ('Numquam', 21)]

So my question is how to access a windowed table from outside stream?

Versions

  • Python version: 3.6.8
  • Faust version: 1.9 (retrieved from pip)
  • Operating system: Ubuntu 16.04.6 LTS
  • Kafka version: Confluent 5.3.1

Thank you,