Official implementation for the SIGGRAPH Asia 2024 paper SPARK: Self-supervised Personalized Real-time Monocular Face Capture
Official implementation for the SIGGRAPH Asia 2024 paper SPARK: Self-supervised Personalized Real-time Monocular Face Capture
SPARK: Self-supervised Personalized Real-time Monocular Face Capture
Kelian Baert
·
Shrisha Bharadwaj
·
Fabien Castan
·
Benoit Maujean
·
Marc Christie
·
Victoria Fernandez Abrevaya
·
Adnane Boukhayma
Technicolor | Max Planck Institute | INRIA Rennes
SIGGRAPH Asia 2024 Conference PapersIf you find our code or paper useful, please cite as:
…Detailsln TrackerAdaptation/submodules/EMOCA/assets/FLAME/geometry/generic_model.pkl MultiFLARE/assets/flame/flame2020.pklgeneric_model.pkl at ./MultiFLARE/assets/flame/flame2020.pkl.TrackerAdaptation/submodules/EMOCA/assets/FLAME/texture/FLAME_albedo_from_BFM.npz.SPARK has been tested with NVIDIA RTX A5000 (24GB) or RTX A4000 (16GB) GPUs. It is possible to train on GPUs with less memory by reducing the batch size.
Please refer to the MonoFaceCompute repository to preprocess your own data.
SPARK is a two-stage approach. First, run MultiFLARE to reconstruct a 3D Face Avatar from multiple videos. Then, use TrackerAdaptation to adapt an existing 3D face tracker to your avatar for real-time tracking through transfer learning.
Detailscd MultiFLARE
python train.py --config configs/example.txt
# Export neutral mesh
python export_mesh.py --config configs/example.txt --resume 3000 --out_dir /tmp/example_mesh --tex_type albedoWe advise starting from the provided example config and modifying input_dir, train_dir and output_dir. For a list of all parameters, please refer to arguments.py or the output of python train.py --help. Parameters can be passed either in the config file or as command line arguments.
…The code in this repository is subject to multiple licenses.
Original Code (Technicolor Group & INRIA Rennes)
Third-Party Code (Max Planck Institute for Intelligent Systems)
./MultiFLARE/flame, ./MultiFLARE/flare, ./TrackerAdaptation/submodulesNo open issues yet, or sync has not completed.