Paper | Project Page | Dataset

Yuekun Dai, Haitian Li, Shangchen Zhou, Chen Change Loy

S-Lab, Nanyang Technological University

Code will be released soon. Please stay tuned.

RGBA images, with the additional alpha channel, are crucial for any application that needs blending, masking, or transparency effects, making them more versatile than standard RGB images. Nevertheless, existing image inpainting methods are designed exclusively for RGB images. Conventional approaches to transparent image inpainting typically involve placing a background underneath RGBA images and employing a two-stage process: image inpainting followed by image matting. This pipeline, however, struggles to preserve transparency consistency in edited regions, and matting can introduce jagged edges along transparency boundaries. To address these challenges, we propose Trans-Adapter, a plug-and-play adapter that enables diffusion-based inpainting models to process transparent images directly. Trans-Adapter also supports controllable editing via ControlNet and can be seamlessly integrated into various community models. To evaluate our method, we introduce LayerBench, along with a novel non-reference alpha edge quality evaluation metric for assessing transparency edge quality.

git clone https://github.com/ykdai/Trans-Adapter.git
cd Trans-Adapter

uv venv trans_adapter --python 3.10

source trans_adapter/bin/activate      # Linux / macOS

uv pip install --upgrade pip setuptools wheel
uv pip install -r requirements.txt

Please download the following files and place them under the ckpt/ directory:

Download and extract to dataset/layerbench/:

unzip dataset/layerbench.zip -d dataset/

Expected structure:

dataset/layerbench/
├── layerbench.csv   # image IDs, mask paths, text prompts
├── imgs/            # 800 RGBA source images
└── mask/            # 800 inpainting masks

Trans-Adapter was trained on a combination of MAGICK and an internal layertrain dataset. It will be updated soon.

Run inference on LayerBench. Two processes run in parallel on 2 GPUs, generating PureNoise (str=1.0) and BlendedNoise (str=0.99) outputs simultaneously:

source .venv/bin/activate

CUDA_VISIBLE_DEVICES=0,1 accelerate launch \
  --num_processes 2 \
  --main_process_port 29513 \
  --mixed_precision fp16 \
  train_sd15_inpaint.py \
  --config trans_adapter/configs/eval_baseline.yaml

Outputs are saved to:

outputs/eval_sd15_...-<TIMESTAMP>/
└── samples/
    ├── sample-eval-str1/     # PureNoise  (strength=1.0)
    └── sample-eval-str099/   # BlendedNoise (strength=0.99)
        ├── rgba/             # Generated RGBA images
        └── blended/          # RGBA composited onto GT background (for visual inspection)

Computes four metrics — Aesthetic Score (AS), LPIPS, CLIP Similarity, and AEQ — on inference outputs. Replace <OUTPUT_DIR> with the timestamped output folder.

CUDA_VISIBLE_DEVICES=0 python eval_metric.py \
  --input_dir <OUTPUT_DIR>/samples/sample-eval-str1 \
  --gt_dir dataset/layerbench/imgs \
  --mask_dir dataset/layerbench/mask \
  --prompts_file dataset/layerbench/layerbench.csv \
  --res 512 --blended

CUDA_VISIBLE_DEVICES=0 python eval_metric.py \
  --input_dir <OUTPUT_DIR>/samples/sample-eval-str099 \
  --gt_dir dataset/layerbench/imgs \
  --mask_dir dataset/layerbench/mask \
  --prompts_file dataset/layerbench/layerbench.csv \
  --res 512 --blended

Results are saved as evaluation_result_sum_blended.csv and evaluation_result_blended.csv in each sample directory.

If you find this work useful, please cite:

@article{dai2025transadapter,
  title     = {Trans-Adapter: A Plug-and-Play Framework for Transparent Image Inpainting},
  author    = {Dai, Yuekun and Li, Haitian and Zhou, Shangchen and Loy, Chen Change},
  journal   = {ICCV},
  year      = {2025},
}

This project builds on AnimateDiff, Stable Diffusion, and BrushNet. We thank the authors for their excellent open-source contributions.

For questions or inquiries, please contact ydai005@e.ntu.edu.sg.