FastBlend: a Model-Free Algorithm That Can Make Your Video Smooth
This is a model-free algorithm that can make your video smooth. You can remove the flicker in your video, or render a fluent video using a series of keyframes.
The original video is here. We only use the first 236 frames.
Re-render each frame independently. The parameters are
Prompt: masterpiece, best quality, anime screencap, cute, petite, long hair, black hair, blue eyes, hoodie, breasts, smile, short sleeves, hands, blue bowknot, wind, depth of field, forest, close-up,
We found an interesting project called CoDeF, which uses only one keyframe to render a video. We collected some videos from their project page and compared FastBlend with CoDeF. For each video, we select one keyframe from the rendered video and use this keyframe to rerender the video in interpolation mode. The parameters are
Batch size: 48
Tracking Window Size: 1
Minimum patch size (odd number, larger is better): 25
Number of iterations: 5
Guide weight: 10
NNF initialization: identity
As we can see, FastBlend is competitive with CoDeF. What's more, FastBlend is very efficiency. We only need one minute to render a video clip!
Reference
Blend
Output directory: the directory to save the video.
Inference mode
Mode
Time
Memory
Quality
Frame by frame output
Description
Fast
■
■■■
■■
No
Blend the frames using a tree-like data structure, which requires much RAM but is fast.
Balanced
■■
■
■■
Yes
Blend the frames naively.
Accurate
■■■
■
■■■
Yes
Blend the frames and align them together for higher video quality. When [batch size] >= [sliding window size] * 2 + 1, the performance is the best.
Sliding window size: our algorithm will blend the frames in a sliding windows. If the size is n, each frame will be blended with the last n frames and the next n frames. A large sliding window can make the video fluent but sometimes smoggy.
Batch size: a larger batch size makes the program faster but requires more VRAM.
Tracking window size (only for accurate mode): The size of window in which our algorithm tracks moving objects. Empirically, 1 is enough.
Advanced settings
Minimum patch size (odd number): the minimum patch size used for patch matching. (Default: 5)
Number of iterations: the number of iterations of patch matching. (Default: 5)
Guide weight: a parameter that determines how much motion feature applied to the style video. (Default: 10)
NNF initialization: how to initialize the NNF (Nearest Neighbor Field). (Default: identity)
Interpolate
Output directory: the directory to save the video.
Batch size: a larger batch size makes the program faster but requires more VRAM.
Tracking window size (only for accurate mode): The size of window in which our algorithm tracks moving objects. Empirically, 1 is enough.
Advanced settings
Minimum patch size (odd number): the minimum patch size used for patch matching. This parameter should be larger than that in blending. (Default: 15)
Number of iterations: the number of iterations of patch matching. (Default: 5)
Guide weight: a parameter that determines how much motion feature applied to the style video. (Default: 10)
NNF initialization: how to initialize the NNF (Nearest Neighbor Field). (Default: identity)
Cite this project
@article{duan2023fastblend,
title={FastBlend: a Powerful Model-Free Toolkit Making Video Stylization Easier},
author={Duan, Zhongjie and Wang, Chengyu and Chen, Cen and Qian, Weining and Huang, Jun and Jin, Mingyi},
journal={arXiv preprint arXiv:2311.09265},
year={2023}
}