#211·dolly

Tensor sizes are printed while running .generate function

Author: chopra-anujCreated Oct 19, 2023Updated Oct 19, 2023

When I run .generate function of any of the 3b/7b/12b model, I get continuously tensor sizes printed which looks like torch.Size([1, 510]) torch.Size([1, 40, 509, 128]) torch.Size([1, 1]) torch.Size([1, 511]) torch.Size([1, 40, 510, 128]) torch.Size([1, 1]) .........

I think in code somewhere print( X.shape ) is written which is creating problem. I tried using Huggingface pipeline also but same problem. I do not encounter this problem while using any other LLM model