Cosmos-Predict2 is a collection of general-purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream ap
Cosmos-Predict2 is a collection of general-purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream ap
Update Notice
The latest version of our Cosmos-Predict is now live!
We recommend all users migrate to the new version for improved performance, features, and continued support.
Cosmos-Predict2 is a key branch of the Cosmos World Foundation Models (WFMs) ecosystem for Physical AI, specializing in future state prediction through advanced world modeling. It offers two powerful capabilities: text-to-image generation for creating high-quality images from text descriptions, and video-to-world generation for producing visual simulations from video inputs.
We visualize the architecture of Cosmos-Predict2 in the following figure.
System Requirements:
We HIGHLY recommend using uv.
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/envCosmos-Predict2 can be installed with pip (requires python==3.10):
uv venv --python 3.10 --allow-existing
uv pip install -U "cosmos-predict2[cu126]" --extra-index-url https://nvidia-cosmos.github.io/cosmos-dependencies/cu126_torch260/simpleTo run the repository examples below, please follow the Setup Guide
Cosmos-Predict2 is included in diffusers>=0.34.0.
Run example inference scripts:
./scripts/hf_text2image.py output/hf_text2image --prompt "assets/text2image/example_prompt.txt" -v./scripts/hf_video2world.py output/hf_video2world --prompt "assets/video2world/example_prompt.txt" --image "assets/video2world/example_input.jpg" -vHere is a quick example demonstrating how to use Cosmos-Predict2-2B-Video2World for video generation:
…Input prompt:
A high-definition video captures the precision of robotic welding in an industrial setting. The first frame showcases a robotic arm, equipped with a welding torch, positioned over a large metal structure. The welding process is in full swing, with bright sparks and intense light illuminating the scene, creating a vivid display of blue and white hues. A significant amount of smoke billows around the welding area, partially obscuring the view but emphasizing the heat and activity. The background reveals parts of the workshop environment, including a ventilation system and various pieces of machinery, indicating a busy and functional industrial workspace. As the video progresses, the robotic arm maintains its steady position, continuing the welding process and moving to its left. The welding torch consistently emits sparks and light, and the smoke continues to rise, diffusing slightly as it moves upward. The metal surface beneath the torch shows ongoing signs of heating and melting. The scene retains its industrial ambiance, with the welding sparks and smoke dominating the visual field, underscoring the ongoing nature of the welding operation.
| Input image | Output video |
|---|---|
Our setup guide provides complete information on
For inference examples and usage
For post-training customization
Our performance guide includes
We thrive on community collaboration! NVIDIA-Cosmos wouldn't be where it is without contributions from developers like you. Check out our Contributing Guide to get started, and share your feedback through issues.
Big thanks to everyone helping us push the boundaries of open-source physical AI!
This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.
This model includes safety and content moderation features powered by Llama Guard 3. Llama Guard 3 is used solely as a content input filter and is subject to its own license.
NVIDIA Cosmos source code is released under the Apache 2 License.
NVIDIA Cosmos models are released under the NVIDIA Open Model License. For a custom license, please contact [email protected].
No open issues yet, or sync has not completed.