add PAG support for sd_1.5 controlnet

[tuanh123789]

What does this PR do?

Adds PAG (Perturbed-Attention Guidance) support for SD 1.5 with controlnet models (StableDiffusionControlNetPAGPipeline)
Continuation of #8710

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@yiyixuxu
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Sample using SD1.5 controlnet	Sample SD1.5 controlnet PAG

from diffusers import AutoPipelineForText2Image, ControlNetModel, UniPCMultistepScheduler
from diffusers.utils import load_image
import numpy as np
import torch

import cv2
from PIL import Image

# download an image
image = load_image(
   "https://hf.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/hf-logo.png"
)
image = np.array(image)

# get canny image
image = cv2.Canny(image, 100, 200)
image = image[:, :, None]
image = np.concatenate([image, image, image], axis=2)
canny_image = Image.fromarray(image)

# load control net and stable diffusion v1-5
controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16)
pipe = AutoPipelineForText2Image.from_pretrained(
   "runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16, enable_pag=True
)

# speed up diffusion process with faster scheduler and memory optimization
# remove following line if xformers is not installed
pipe.enable_xformers_memory_efficient_attention()

pipe.enable_model_cpu_offload()

# generate image
generator = torch.manual_seed(0)
image = pipe(
   "aerial view, a futuristic research complex in a bright foggy jungle, hard lighting",
   guidance_scale=7.5,
   generator=generator,
   image=canny_image,
   pag_scale=10,
).images[0]

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[a-r-r-o-w]

thank you for taking this up! it's looking good to me for the most part but requesting a few minor changes and then this should be ready for merge. also need you to rebase/merge your branch on latest diffusers:main correctly in order for the SD3 Inpaint pipeline changes to reflect because they are all deleted at the moment

[a-r-r-o-w]

Why has this been removed? Could you rebase/cherrypick your branch on current diffusers main?

[a-r-r-o-w]

Need to revert these changes

[a-r-r-o-w]

Will need to revert these changes and re-run make fix-copies

[a-r-r-o-w]

nice!

[a-r-r-o-w]

This method is deprecated and therefore can be removed from here

[a-r-r-o-w]

Suggested change

	callback (`Callable`, *optional*):
	A function that calls every `callback_steps` steps during inference. The function is called with the
	following arguments: `callback(step: int, timestep: int, latents: torch.Tensor)`.
	callback_steps (`int`, *optional*, defaults to 1):
	The frequency at which the `callback` function is called. If not specified, the callback is called at
	every step.

[a-r-r-o-w]

Suggested change

	callback = kwargs.pop("callback", None)
	callback_steps = kwargs.pop("callback_steps", None)
	if callback is not None:
	deprecate(
	"callback",
	"1.0.0",
	"Passing `callback` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
	)
	if callback_steps is not None:
	deprecate(
	"callback_steps",
	"1.0.0",
	"Passing `callback_steps` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
	)

These have been deprecated and we do not need to have them in newer pipelines. You'll have to make the necessary changes elsewhere too to remove any usage occurences

[a-r-r-o-w]

IP Adapter image embeds need to be prepared differently for perturbed attention guidance. Please take a look here:

diffusers/src/diffusers/pipelines/pag/pipeline_pag_sd_xl.py

Line 1156 in a785992

if ip_adapter_image is not None or ip_adapter_image_embeds is not None:

[a-r-r-o-w]

Oh, I see you've done what I asked in previous comment here. Is there a reason to do it separately? If not, can you move it under the same if-branch?

[a-r-r-o-w]

Suggested change

	if callback is not None and i % callback_steps == 0:
	step_idx = i // getattr(self.scheduler, "order", 1)
	callback(step_idx, t, latents)

it's been deprecated