PhysFluids

Official implementation of Learning an Implicit Physics Model for Image-based Fluid Simulation, accepted at ICCV 2025.

[Paper] [Project page]

Clone the repository and all submodules:

git clone --recursive https://github.com/emily-jia/physfluids.git
cd physfluids

If the repository was cloned without --recursive, run:

git submodule update --init --recursive

We recommend Linux with an NVIDIA GPU. First install PyTorch, torchvision, PyTorch3D, and torch-cluster versions compatible with your CUDA toolkit. Then install the remaining Python dependencies:

pip install -r requirements.txt

Install the CUDA extensions from the cloned submodules:

pip install --no-build-isolation \
    ./gaussian_splatting/submodules/simple-knn

pip install --no-build-isolation \
    ./gaussian_splatting/submodules/diff-gaussian-rasterization

pip install --no-build-isolation \
    ./third_party/diff-gaussian-rasterization

gaussian_splatting/SIBR_viewers is used only by the optional interactive viewer and is not required for training or the demo. Refer to the upstream Gaussian Splatting documentation if you want to build it.

The renderer in third_party/diff-gaussian-rasterization is imported as diff_gauss and supports the extra per-Gaussian attributes used by train_utils/render_feats.py. The stock diff_gaussian_rasterization extension included with Gaussian Splatting does not provide this feature interface and is not a substitute for diff_gauss.

Large model weights are distributed separately and are ignored by Git.

First, download the pretrained depth and inpainting models from the 3D Cinemagraphy checkpoint release. Extract the archive and place these three files under ckpts/:

ckpts/
|-- color-model.pth
|-- depth-model.pth
`-- dpt_hybrid-midas-501f0c75.pt

Then download our released motion and warping checkpoints into exps/motion_refine/:

mkdir -p exps/motion_refine
gdown --folder \
    https://drive.google.com/drive/folders/14yIHl95qHyBkLBQRbFoXPauCb0itfTEs \
    -O exps/motion_refine

An example input is included under demo/0. Run:

python demo_warping.py \
    --config configs/config_motion_only.yaml \
    --input_dir demo/0 \
    --ckpt_path exps/motion_refine/model_230000.pth \
    --flow_scale 1.0 \
    --ds_factor 1.0

To prepare a new input and create region masks and motion hints with LabelMe, follow the 3D Cinemagraphy usage instructions.

Training uses the Eulerian fluid dataset. Set data.path in the YAML files to the dataset root before launching a job. The expected root contains train/validation videos and depth maps, motion hints, optical flow, edge masks, camera poses, and index JSON files.

python train_motion_only.py \
    --config configs/config_motion_only.yaml \
    --num_workers 8

Outputs are saved under exps/<exp_name>/. With the released config this is exps/motion_refine/, and motion checkpoints are named:

motion_<step>.pth

If a checkpoint already exists in that directory, training resumes from the latest one unless --no_reload is provided.

train_warping.py expects both:

1. a pretrained animation-model checkpoint passed through --ckpt_path;
2. a motion_*.pth checkpoint inside exps/<exp_name>/ from the selected warping config.

For example:

mkdir -p exps/decoder_lr=1e-2
cp /path/to/motion_120000.pth exps/decoder_lr=1e-2/

python train_warping.py \
    --config configs/config_warping.yaml \
    --ckpt_path /path/to/model_150000.pth \
    --num_workers 8

Warping checkpoints are written as:

exps/<exp_name>/model_<step>.pth

Both training scripts currently support batch size 1. Use --distributed only when launching them through an appropriately configured distributed PyTorch environment.

This codebase is developed based on 3D Cinemagraphy from a Single Image.

If you find this project useful, please cite:

@InProceedings{Jia_2025_ICCV,
    author    = {Jia, Emily Yue-Ting and Mao, Jiageng and Gao, Zhiyuan and Zhao, Yajie and Wang, Yue},
    title     = {Learning an Implicit Physics Model for Image-based Fluid Simulation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2025},
    pages     = {7048--7057}
}