Constant time with AutoShape

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi,

I wanted to process 4K Images with yolov5s and did some experiments regarding resolution and size.

I realized I get the best result of my network with images of input size 640x640 and splitting the 4k images, respectively.

For inference, I now did some experiments and I am not even close to 30 FPS when processing with AutoShape.

This is probably due to the preprossessing of AutoShape, yet I do not find a way around that (neither with multiprocessing/treading or anything I can currently think of). I am quite sure that the problem is in AutoShape and wanted to ask if you have any idea how to accelerate that.

My current experiments show that if you are passing a torch tensor through the network, inference time seems to be more constant.

`import torch import numpy import time

model = torch.hub.load('ultralytics/yolov5', "yolov5s", classes = 10) model.eval()

input_image = numpy.random.rand(3840, 2160,3) def split_images(img): dh, dw, _ = img.shape image_splits =[] height, width = 640, 640 for i in range(dh//height): for j in range(dw//width): image_splits.append( (jwidth, iheight, (j+1)*width, (i+1)*height)) return image_splits

splits = split_images(input_image) warmup = model([input_image]) t = time.time() model([input_image]) print(f"Time for completing Single Frame: {time.time()-t}") cropped = [] for i in splits: cropped.append(input_image[i[1]:i[3],i[0]:i[2]])

t = time.time() model(cropped) print(f"Time for completing 18 Cropped Frames: {time.time()-t}")

inp = torch.rand(18,3,640,640) t = time.time() model(inp) print(f"Time for completing 18 Torch Frames: {time.time()-t}")` Result: ... Adding AutoShape... Time for completing Single Frame: 1.4259703159332275 Time for completing 18 Cropped Frames: 0.5461986064910889 Time for completing 18 Torch Frames: 0.027060985565185547

Additional

No response