[LoRA] support Flux Control LoRA with bnb 8bit.

[sayakpaul]

What does this PR do?

Fixes #11648

Code to test:

import torch
from diffusers import FluxControlPipeline
from image_gen_aux import DepthPreprocessor
from diffusers.utils import load_image
from diffusers.quantizers import PipelineQuantizationConfig

pipeline = FluxControlPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    quantization_config=PipelineQuantizationConfig(
        quant_backend="bitsandbytes_8bit", 
        quant_kwargs={"load_in_8bit": True}, 
        components_to_quantize=["transformer", "text_encoder_2"]
    ),
    torch_dtype=torch.float16,
).to("cuda")
pipeline.load_lora_weights("black-forest-labs/FLUX.1-Depth-dev-lora")

prompt = "A robot made of exotic candies and chocolates of different kinds. The background is filled with confetti and celebratory gifts."
control_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/robot.png")

processor = DepthPreprocessor.from_pretrained("LiheYoung/depth-anything-large-hf")
control_image = processor(control_image)[0].convert("RGB")

image = pipeline(
    prompt=prompt,
    control_image=control_image,
    height=1024,
    width=1024,
    num_inference_steps=30,
    guidance_scale=10.0,
    generator=torch.Generator().manual_seed(42),
).images[0]
image.save("output.png")

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.