Zeqi Xiao Yifan Zhou Shuai Yang Xingang Pan
Install the environments by
conda create -n moft python==3.8
conda activate moft
pip install -r requirements.txt
Downloads checkpoints from Animatediff, LoRA, and SD-1.5. Put them into the following structures:
models/
├── DreamBooth_LoRA
│ ├── realisticVisionV20_v20.safetensors
├── Motion_Module
│ ├── mm_sd_v15_v2.ckpt
├── stable-diffusion-v1-5
Run process.ipynb
If you find our work helpful, please cite:
@inproceedings{
xiao2024video,
title={Video Diffusion Models are Training-free Motion Interpreter and Controller},
author={Zeqi Xiao and Yifan Zhou and Shuai Yang and Xingang Pan},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=ZvQ4Bn75kN}
}