arror at Step3: Track masks and get the inpainting result
error happens both whel I run local and when I run In HugingFace at same step.
Initializing BaseSegmenter to cuda Pretrained flow completion model has loaded... Pretrained ProPainter has loaded... Network [InpaintGenerator] was created. Total number of parameters: 39.4 million. To see the architecture, do print(network).
- Running on local URL: http://127.0.0.1:7860/
- Running on public URL: https://1ca202b21111e6ed74.gradio.live/
This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run gradio deploy from the terminal in the working directory to deploy to Hugging Face Spaces (https://huggingface.co/spaces)
Tracking image: 100% 49/49 [00:04<00:00, 12.01it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 242, in vos_tracking_video
video_output = generate_video_from_frames(video_state["painted_images"], output_path="./result/track/{}".format(video_state["video_name"]), fps=fps) # import video_input to name the output video
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 310, in generate_video_from_frames
torchvision.io.write_video(output_path, frames, fps=fps, video_codec="libx264")
File "/usr/local/lib/python3.11/dist-packages/torchvision/io/video.py", line 121, in write_video
stream = container.add_stream(video_codec, rate=fps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/container/output.pyx", line 85, in av.container.output.OutputContainer.add_stream
File "av/utils.pyx", line 51, in av.utils.to_avrational
AttributeError: 'numpy.float64' object has no attribute 'numerator'
Tracking image: 100% 49/49 [00:01<00:00, 27.57it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 242, in vos_tracking_video
video_output = generate_video_from_frames(video_state["painted_images"], output_path="./result/track/{}".format(video_state["video_name"]), fps=fps) # import video_input to name the output video
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 310, in generate_video_from_frames
torchvision.io.write_video(output_path, frames, fps=fps, video_codec="libx264")
File "/usr/local/lib/python3.11/dist-packages/torchvision/io/video.py", line 121, in write_video
stream = container.add_stream(video_codec, rate=fps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/container/output.pyx", line 85, in av.container.output.OutputContainer.add_stream
File "av/utils.pyx", line 51, in av.utils.to_avrational
AttributeError: 'numpy.float64' object has no attribute 'numerator'
Tracking image: 100% 49/49 [00:01<00:00, 27.39it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 242, in vos_tracking_video
video_output = generate_video_from_frames(video_state["painted_images"], output_path="./result/track/{}".format(video_state["video_name"]), fps=fps) # import video_input to name the output video
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 310, in generate_video_from_frames
torchvision.io.write_video(output_path, frames, fps=fps, video_codec="libx264")
File "/usr/local/lib/python3.11/dist-packages/torchvision/io/video.py", line 121, in write_video
stream = container.add_stream(video_codec, rate=fps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/container/output.pyx", line 85, in av.container.output.OutputContainer.add_stream
File "av/utils.pyx", line 51, in av.utils.to_avrational
AttributeError: 'numpy.float64' object has no attribute 'numerator'
100% 10/10 [00:04<00:00, 2.05it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 293, in inpaint_video
video_output = generate_video_from_frames(inpainted_frames, output_path="./result/inpaint/{}".format(video_state["video_name"]), fps=fps) # import video_input to name the output video
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 310, in generate_video_from_frames
torchvision.io.write_video(output_path, frames, fps=fps, video_codec="libx264")
File "/usr/local/lib/python3.11/dist-packages/torchvision/io/video.py", line 121, in write_video
stream = container.add_stream(video_codec, rate=fps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "av/container/output.pyx", line 85, in av.container.output.OutputContainer.add_stream
File "av/utils.pyx", line 51, in av.utils.to_avrational
AttributeError: 'numpy.float64' object has no attribute 'numerator'
Tracking image: 0% 0/49 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 229, in vos_tracking_video
masks, logits, painted_images = model.generator(images=following_frames, template_mask=template_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/track_anything.py", line 26, in generator
mask, logit, painted_image = self.cutie.track(images[i], template_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/base_tracker.py", line 78, in track
probs = self.tracker.step(frame_tensor, mask, labels) # logits 2 (bg fg) H W
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/inference/inference_core.py", line 309, in step
self._add_memory(image,
File "/content/ProPainter/web-demos/hugging_face/tracker/inference/inference_core.py", line 103, in _add_memory
msk_value, sensory, obj_value, self.obj_logits = self.network.encode_mask(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/model/cutie.py", line 74, in encode_mask
mask_value, new_sensory = self.mask_encoder(image,
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/model/big_modules.py", line 174, in forward
new_sensory = self.sensory_update(g_chunk, sensory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/model/modules.py", line 82, in forward
values = self.transform(torch.cat([g, h], dim=2))
^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 1 but got size 2 for tensor number 1 in the list.
Tracking image: 0% 0/49 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/utils.py", line 890, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/app.py", line 229, in vos_tracking_video
masks, logits, painted_images = model.generator(images=following_frames, template_mask=template_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/track_anything.py", line 26, in generator
mask, logit, painted_image = self.cutie.track(images[i], template_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/base_tracker.py", line 78, in track
probs = self.tracker.step(frame_tensor, mask, labels) # logits 2 (bg fg) H W
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/inference/inference_core.py", line 262, in step
corresponding_tmp_ids, _ = self.object_manager.add_new_objects(objects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/inference/object_manager.py", line 35, in add_new_objects
if obj in self.obj_to_tmp_id:
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/ProPainter/web-demos/hugging_face/tracker/inference/object_info.py", line 21, in eq
return self.id == other.id
^^^^^^^^
AttributeError: 'range' object has no attribute 'id'
Tracking image: 0% 0/49 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 2137, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/gradio/blocks.py", line 1663, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_t
Source: sczhou/ProPainter