Repository navigation
img2img: outputs paired with the wrong init image #15176
Description
Describe the bug
With one init image per prompt and num_images_per_prompt > 1, half of the outputs start from the other prompt's image, and nothing errors. With prompt=["a cat", "a dog"], image=[cat_img, dog_img] and num_images_per_prompt=2, the second "a cat" output starts from dog_img and the first "a dog" output starts from cat_img.
The cause is in StableDiffusionImg2ImgPipeline.prepare_latents. The prompt embeddings are ordered [a, a, b, b], but torch.cat([image] * n) and torch.cat([init_latents] * n) give [a, b, a, b], so rows 1 and 2 get the wrong image.
I confirmed this in SD, SDXL, Kolors and LCM img2img. The same code is copied into 7 more pipelines, which I didn't test. The tests miss it because test_num_images_per_prompt only checks the shape and test_inference_batch_single_identical only compares row 0.
By the way I already have a fix for this. It repeats each image with repeat_interleave when there's one image per prompt, so a single shared image gives the same results as before, and there's a test that fails on main and passes with it. I would love to open a PR if that works.
Reproduction
import torch
from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("hf-internal-testing/tiny-stable-diffusion-torch", safety_checker=None)
pipe.set_progress_bar_config(disable=True)
image_a = torch.rand(1, 3, 64, 64, generator=torch.Generator().manual_seed(1))
image_b = torch.rand(1, 3, 64, 64, generator=torch.Generator().manual_seed(2))
kwargs = dict(num_inference_steps=2, strength=0.5, output_type="pt")
def gen(seed):
return torch.Generator().manual_seed(seed)
# one image per prompt, two outputs per prompt -> expected order: (a, img a), (a, img a), (b, img b), (b, img b)
batched = pipe(["a cat", "a dog"], image=torch.cat([image_a, image_b]), num_images_per_prompt=2,
generator=[gen(i) for i in range(4)], **kwargs).images
for i, (prompt, image) in enumerate([("a cat", image_a), ("a cat", image_a), ("a dog", image_b), ("a dog", image_b)]):
other = image_b if image is image_a else image_a
own = pipe(prompt, image=image, generator=gen(i), **kwargs).images[0]
wrong = pipe(prompt, image=other, generator=gen(i), **kwargs).images[0]
print(f"output {i}: diff vs its own image {(batched[i] - own).abs().max():.2e}, "
f"vs the other prompt's image {(batched[i] - wrong).abs().max():.2e}")Logs
Outputs 1 and 2 match a single call that uses the other prompt's image:
output 0: diff vs its own image 1.04e-05, vs the other prompt's image 4.96e-01
output 1: diff vs its own image 5.31e-01, vs the other prompt's image 8.46e-06
output 2: diff vs its own image 6.56e-01, vs the other prompt's image 1.05e-05
output 3: diff vs its own image 1.08e-05, vs the other prompt's image 5.91e-01System Info
- Diffusers version: 0.42.0.dev0 (main)
- Platform: macOS-15.5-arm64-arm-64bit
- Running on Google Colab?: No
- Python version: 3.12.13
- PyTorch version (GPU?): 2.15.0 (False)
- Huggingface_hub version: 2.4.0
- Transformers version: 5.23.0.dev0
- Accelerate version: 1.15.0
- PEFT version: 0.21.2
- Safetensors version: 0.8.0
- xFormers version: not installed
- Accelerator: Apple M4
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
Activity
Arnav66692 commentedon Nov 3, 2026
By the way I already have a fix for this and would love to open a PR if that works