Stable Diffusion XL is a latent diffusion image model whose published base checkpoint can run alone or feed an optional refiner for the final denoising steps. This guide follows the official Stability AI model card and keeps every performance recommendation measurable.
1. Create a pinned environment
Start with the packages named by the model card, then record the resolved versions in your project lockfile:
python -m pip install --upgrade diffusers transformers accelerate safetensors invisible_watermark
Pin PyTorch and CUDA versions separately for the target hardware. Test upgrades against a fixed prompt-and-seed evaluation set before rollout.
2. Run the SDXL base checkpoint
import torch
from diffusers import DiffusionPipeline
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
pipe = DiffusionPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True,
)
pipe.to("cuda")
generator = torch.Generator(device="cuda").manual_seed(42)
image = pipe(
prompt="A product photograph of a ceramic tea cup on slate, soft window light",
negative_prompt="blurry, distorted, illegible text",
num_inference_steps=30,
generator=generator,
).images[0]
image.save("sdxl-base.png")
Record the prompt, negative prompt, seed, resolution, scheduler, inference steps, dtype, device, and checkpoint revision with every benchmark.
3. Use CPU offload when GPU memory is constrained
The official model card recommends model CPU offload as an alternative to moving the whole pipeline to CUDA:
# Replace pipe.to("cuda") with:
pipe.enable_model_cpu_offload()
Offload can reduce peak GPU memory while increasing latency and host-memory traffic. Measure both peak memory and end-to-end generation time on the deployment hardware.
4. Add the optional refiner only after an evaluation
The refiner is a separate checkpoint. It consumes the base model's latent output for the final denoising portion. Load both in reduced precision, reuse compatible components where documented, and compare the two-stage result with base-only output on the same prompts.
refiner = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-refiner-1.0",
text_encoder_2=pipe.text_encoder_2,
vae=pipe.vae,
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True,
)
refiner.to("cuda")
Do not assume the refiner is always worth its additional load time, memory, and denoising work. Measure preference or task success with representative images.
5. Benchmark the pipeline, not the model name
Track at least:
- prompt adherence and compositional accuracy
- text rendering, faces, hands, and known failure modes
- latency and images per minute at the target concurrency
- peak GPU and host memory
- safety-filter behavior and review workload
- reproducibility across library, driver, and checkpoint updates
Compare base-only SDXL, base-plus-refiner SDXL, and newer image models such as MAI-Image-2.5-Pro or Qwen-Image-3.0 on the same production prompts.
6. Complete the deployment review
The SDXL model card calls out limitations involving photorealism, legible text, composition, faces, lossy autoencoding, and bias. Add deployment-specific content controls, watermarking decisions, license review, abuse monitoring, and human escalation before exposing generation to users.