GenPIE

Research code release for GenPIE: A Time-Resolved Plenoptic Imager (ACM Transactions on Graphics, 2026).

GenPIE reconstructs explicit scene geometry, principled materials, and illumination from multi-view transient measurements, then learns a ray-conditioned temporal compensation kernel. This release also includes the paper's multi-bounce decomposition and time-unwarping operators.

conda env create -f environment.yml
conda activate genpie

Equivalent pip installation in an existing Python environment:

python -m pip install -e ".[rendering,evaluation,dev]"

Run the test suite:

python -m pytest -q

Inverse rendering is designed for the cuda_ad_mono Mitsuba variant. The neural compensation and array-only application modules can be tested on CPU.

configs/                 Explicit inverse-rendering and compensation configs
src/genpie/
  applications/          Bounce decomposition and time unwarping
  calibration/           Camera-to-world validation and ray geometry
  cli/                   Config-driven training entry points
  data/                  HDF5 I/O, streaming bundler, and datasets
  evaluation/            Transient and mesh metrics
  models/                Neural compensation and losses
  optimization/          Inverse-rendering and compensation trainers
  rendering/             Mitsuba/mitransient multi-view renderer
tests/                    CPU unit and integration tests

All HDF5 arrays use the root key dataset. Per-view transient volumes use (H, W, T, C); bundled ray samples use (R, T, C). Camera extrinsics are camera-to-world matrices with shape (N, 4, 4). Neural compensation consumes first-hit world positions (R, 3) and surface-to-camera view directions encoded as [theta, phi] with shape (R, 2).

Read Data Format before preparing measurements. In particular, the forward renderer does not infer metric first-hit positions; those must come from calibrated geometry or a metric ray-distance map.

If you think this work is interesting and helpful, you can cite our work through:

@article{wang2026genpie,
  title   = {GenPIE: A Time-Resolved Plenoptic Imager},
  author  = {Wang, Ziheng and Shen, Siyuan and Xu, Huanyu and Qiao, Kaichun and Zhang, Longwen and Zhang, Qixuan and Sun, Qilin and Li, Shiying and Yu, Jingyi},
  journal = {ACM Transactions on Graphics},
  volume  = {45},
  number  = {4},
  year    = {2026},
  doi     = {10.1145/3811281}
}