Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Linear Image Generation by Synthesizing Exposure Brackets

Accepted to CVPR 2026.

Project Page | Paper | Video

Image

This repository provides the official implementation for the following paper:

Linear Image Generation by Synthesizing Exposure Brackets
Accepted to CVPR 2026
arXiv

Abstract

The life of a photo begins with photons striking the sensor, whose signals are passed through a sophisticated image signal processing (ISP) pipeline to produce a display-referred image. However, such images are no longer faithful to the incident light, being compressed in dynamic range and stylized by subjective preferences. In contrast, RAW images record direct sensor signals before non-linear tone mapping. After camera response curve correction and demosaicing, they can be converted into linear images, which are scene-referred representations that directly reflect true irradiance and are invariant to sensor-specific factors. Since image sensors have better dynamic range and bit depth, linear images contain richer information than display-referred ones, leaving users more room for editing during post-processing. Despite this advantage, current generative models mainly synthesize display-referred images, which inherently limits downstream editing. In this paper, we address the task of text-to-linear-image generation: synthesizing a high-quality, scene-referred linear image that preserves full dynamic range, conditioned on a text prompt, for professional post-processing. Generating linear images is challenging, as pre-trained VAEs in latent diffusion models struggle to simultaneously preserve extreme highlights and shadows due to the higher dynamic range and bit depth. To this end, we represent a linear image as a sequence of exposure brackets, each capturing a specific portion of the dynamic range, and propose a DiT-based flow-matching architecture for text-conditioned exposure bracket generation. We further demonstrate downstream applications including text-guided linear image editing and structure-conditioned generation via ControlNet.


1. Environment

pip install uv
uv venv ~/.venv/lineargen && source ~/.venv/lineargen/bin/activate
uv pip install torch
uv pip install -r requirements.txt

Notes:

  • lineargen/ is a local (vendored) package — run all scripts from the repo root so import lineargen... resolves.
  • The clip line in requirements.txt is an unrelated PyPI CLI tool and is not imported by this code; CLIP functionality comes from open-clip-torch. It is safe to skip.
  • Full environment / FLUX checkpoint setup notes: see setup.md.

2. Model & data

  • Base model: black-forest-labs/FLUX.1-dev (gated on Hugging Face). Set HF_HOME to a cache that contains it, or huggingface-cli download black-forest-labs/FLUX.1-dev.

  • Datasets:

    Dataset Role Link
    RAISE Training Link
    MIT5K Test set Google Drive
  • Training data: captions CSV with raw_path, xmp_path, caption columns, read by lineargen/data/mit5k_dataset.py; point train_data.root_dir / train_data.csv_path in the config at your data. Each RAW is auto-exposed and decomposed into multi-EV frames (evs: [-4,-2,0,2]).

    The paper's training data can't be released under Adobe's license. The released checkpoint is trained on a smaller, reproduced set instead, so its results may differ from the paper's numbers — treat it as a reference implementation, not a metrics reproduction. For access to additional data, please contact ydai005@e.ntu.edu.sg.

  • Test set: full MIT5K (dataset_path/mit5k_captions.csv, includes a log_lum GT column). Config: lineargen/configs/eval/eval_lineargen_baseline_lum_denoise.yaml; metrics: calculate_metrics_mit5k.py (FID / aesthetic / NIQE / CLIP-sim / luminance-scale / log-lum MAE).

3. Training

Baseline (image generation only, no lum-token denoise):

CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch --main_process_port 29502 --mixed_precision bf16 \
  train_lineargen_inflated_flux_mit5k.py \
  --config lineargen/configs/train/train_lineargen_baseline.yaml

With the lum-token denoise branch (adds luminance prediction via flow-matching + a terminal attention-pool head; gated by inflation_kwargs.use_lum_token / use_lum_attn_cls in the config):

CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch --main_process_port 29502 --mixed_precision bf16 \
  train_lineargen_inflated_flux_mit5k_lumtoken_denoise.py \
  --config lineargen/configs/train/train_lineargen_baseline_lum_denoise.yaml

Checkpoints are written to outputs/<config-name>-<timestamp>/checkpoints/.

4. Inference

One eval script serves both models. Set pretrained_lora_path in the config to your .ckpt:

CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch --num_processes=4 --multi_gpu \
  --main_process_port 29526 --mixed_precision bf16 \
  eval_lineargen_inflated_flux_mit5k.py \
  --config lineargen/configs/eval/eval_lineargen_baseline.yaml \
  --output_dir evals/baseline

Or via the helper: ./test_multi_gpu_inference.sh <config.yaml> [num_gpus]. With a lum-token-denoise checkpoint, the pipeline additionally returns lum_attn_cls_log_radiance per prompt from the terminal attention-pool head. For the full-MIT5K luminance evaluation use lineargen/configs/eval/eval_lineargen_baseline_lum_denoise.yaml and score with calculate_metrics_mit5k.py (see §2).

Running the released checkpoint. Download the released .ckpt (Google Drive) to ckpt/lineargen.ckpt. The provided config lineargen/configs/eval/eval_lineargen_baseline.yaml already points at it, so the command above runs it as-is. Single-GPU quick test:

CUDA_VISIBLE_DEVICES=0 accelerate launch --num_processes=1 --mixed_precision bf16 \
  eval_lineargen_inflated_flux_mit5k.py \
  --config lineargen/configs/eval/eval_lineargen_baseline.yaml \
  --output_dir evals/released

See setup.md §3 for checkpoint placement and the config-must-match-checkpoint rules.

5. AVIF rendering

Render generated linear-radiance outputs to AVIF (HDR-capable):

python save_avif.py outputs/<run>/results/sample_0/ --mode concat

save_avif.py needs a pillow-heif built with an AVIF encoder — see the libheif build notes in setup.md if RuntimeError: No AVIF encoder found. appears.

6. Repository layout

train_lineargen_inflated_flux_mit5k.py                   # baseline train (no lum-token denoise)
train_lineargen_inflated_flux_mit5k_lumtoken_denoise.py  # lum-token-denoise train
eval_lineargen_inflated_flux_mit5k.py                    # inference for both
calculate_metrics_mit5k.py                               # full-MIT5K metrics (FID/aesthetic/NIQE/CLIP/LS/log-lum MAE)
save_avif.py                                             # AVIF rendering
ckpt/                                                    # place the released checkpoint here
lineargen/
  archs_lineargen_flux/    # inflated Flux transformer (baseline + lum-token denoise), motion module
  pipelines_lineargen/     # inference pipelines (baseline + lum-token denoise) + noise init
  data/                    # mit5k dataset, apple-log, gain-map
  utils/                   # bracketing loss
  configs/                 # train/ (training configs) and eval/ (inference/eval configs)
hdr_conversion/            # color / ICC helpers for AVIF
metrics/                   # optional validation metrics
assets/                    # teaser and figures

Citation

If you find this work useful, please cite:

@article{lineargen2026,
  title     = {Linear Image Generation by Synthesizing Exposure Brackets},
  author    = {Dai, Yuekun and Zhang, Zhoutong and Zhou, Shangchen and Zhao, Nanxuan},
  journal   = {CVPR},
  year      = {2026},
}

About

Implementation of "Linear Image Generation by Synthesizing Exposure Brackets"

Resources

Stars

15 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages