TaskMatrix

工具介绍

TaskMatrix TaskMatrix连接了ChatGPT和一系列可视基础模型,以便在聊天时发送和接收图像. 看到没..

核心特点

• •Firstly, run python visual_chatgpt.py --load "Text2Box_cuda:0,Segmenting_cuda:0,Inpainting_cuda:0,ImageCaptioning_cuda:0"
• •Then, say find xxx in the image or segment xxx in the image. xxx is an object. TaskMatrix will return the detection or segmentation result!
• •Now TaskMatrix can support Chinese! Thanks to @Wang-Xiaodong1899 for his efforts.
• •We propose the template idea in TaskMatrix!
• •A template is a pre-defined execution flow that assists ChatGPT in assembling complex tasks involving multiple foundation models.
• •A template contains the experiential solution to complex tasks as determined by humans.
• •A template can invoke multiple foundation models or even establish a new ChatGPT session
• •To define a template, simply adding a class with attributes template_model = True
• •Thanks to @ShengmingYin and @thebestannie for providing a template example in InfinityOutPainting class (see the following gif)
• •Firstly, run python visual_chatgpt.py --load "Inpainting_cuda:0,ImageCaptioning_cuda:0,VisualQuestionAnswering_cuda:0"