#748·nanoGPT

Realtime response generation simulation instead of one big response at the end.

Author: prethivCreated Jul 31, 2026Updated Jul 31, 2026

Currently response generation happening via generate method which do all the text auto regressive generation and outputs only one big text out at the end.

It will be better if we have ability to stream responses .