Accessing windowed table outside stream
Author: orinciogCreated Dec 10, 2019Updated Jan 8, 2025
Hello!
I need to access a windowed table from outside stream. (The reason is that I need to obtain Top K users in a time interval. For this I store the count for each user in a tumbling table - key is the user and the value is the count of that user).
Using the following code,
table_users = app.Table('users_tumbling', default=int,partitions=1) \
.tumbling(timedelta(hours=1),key_index=True,expires=timedelta(hours=10)) \
.relative_to_now()
@app.timer(5.0,on_leader=True)
async def publish_users_tumbling():
logger.info(f"TIMER: {list(table_users.items())}" )I get a list of users with 0 count: TIMER: [('Dolorem', 0), ('Velit', 0), ('Numquam', 0)]
- I use
relative_to_now()because with the default option, I got aRuntimeError: Operation outside of stream iteration
Using a simple table,
table_all=app.Table('users_all',default=int,partitions=1)
@app.timer(5.0,on_leader=True)
async def publish_users_all():
logger.info(f"TIMER: {list(table_all.items())}" )I get all counts: TIMER: [('Dolorem', 95), ('Velit', 32), ('Numquam', 21)]
So my question is how to access a windowed table from outside stream?
Versions
- Python version: 3.6.8
- Faust version: 1.9 (retrieved from pip)
- Operating system: Ubuntu 16.04.6 LTS
- Kafka version: Confluent 5.3.1
Thank you,
Source: robinhood/faust