How to Run Stable Diffusion Locally: ComfyUI and AUTOMATIC1111 Guide

Hardware Requirements and GPU Compatibility

Before installing, verify your GPU meets the minimum requirements. Generation speed scales significantly with VRAM.

Check Your GPU

# NVIDIA
nvidia-smi

# AMD Linux
rocm-smi

# Apple Silicon (just run the macOS installer — Metal support is built in)
system_profiler SPDisplaysDataType | grep VRAM

Operating System Compatibility

Install ComfyUI (Recommended)

ComfyUI is a node-based workflow editor for Stable Diffusion. It has a steeper initial learning curve but is more powerful and resource-efficient than A1111. Most advanced users and professionals use ComfyUI for production workflows.

Step 1: Clone the ComfyUI Repository

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

Step 2: Create a Python Virtual Environment

python -m venv venv

# Activate on Linux/macOS
source venv/bin/activate

# Activate on Windows
venv\Scripts\activate

Step 3: Install PyTorch

Install the correct PyTorch version for your hardware. Go to pytorch.org/get-started for the exact command for your system, or use one of these:

# NVIDIA GPU (CUDA 12.1)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# AMD GPU (ROCm 6.0 — Linux only)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0

# Apple Silicon / CPU
pip install torch torchvision torchaudio

Step 4: Install ComfyUI Dependencies

pip install -r requirements.txt

Step 5: Download a Model

ComfyUI reads models from the `models/checkpoints/` directory. Download a base model:

# Create checkpoints directory
mkdir -p models/checkpoints

# Download SDXL base model (~6.5 GB) -- replace with your preferred download method
# Option 1: Using wget
wget -O models/checkpoints/sd_xl_base_1.0.safetensors \
  https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

# Option 2: Using the Hugging Face CLI (pip install huggingface_hub)
huggingface-cli download stabilityai/stable-diffusion-xl-base-1.0 \
  sd_xl_base_1.0.safetensors --local-dir models/checkpoints/

Step 6: Start ComfyUI

python main.py

Open your browser at `http://127.0.0.1:8188`. You should see the ComfyUI canvas with a default text-to-image workflow already loaded.

Install AUTOMATIC1111 (WebUI)

AUTOMATIC1111 (A1111) provides a traditional web form interface for Stable Diffusion. It has a large extension ecosystem and is easier for beginners who prefer settings panels over node graphs.

Step 1: Clone the Repository

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
cd stable-diffusion-webui

Step 2: Place Your Model

Copy or move your `.safetensors` or `.ckpt` model file into:

stable-diffusion-webui/models/Stable-diffusion/

Step 3: Run the WebUI

**Linux/macOS:**

./webui.sh

**Windows:**

webui-user.bat

The first launch takes 5-15 minutes to download dependencies and set up the virtual environment automatically. Subsequent launches are much faster.

Once started, open: `http://127.0.0.1:7860`

Step 4: Generate Your First Image

1. Type a prompt in the "Positive prompt" box: `a photorealistic cat sitting on a wooden table, natural light, 4k` 2. Type a negative prompt: `blurry, low quality, cartoon, watermark` 3. Set Width and Height to 1024x1024 for SDXL models 4. Click **Generate**

Finding and Downloading Models

Stable Diffusion models are checkpoint files (.safetensors) that define the visual style and capabilities of the generated images. The main repositories are Hugging Face and CivitAI.

Recommended Base Models

Where to Put Downloaded Models

For ComfyUI, place models in these directories by type:

ComfyUI/
  models/
    checkpoints/   ← main model files (.safetensors)
    loras/         ← LoRA fine-tuning weights
    vae/           ← VAE files
    controlnet/    ← ControlNet models
    embeddings/    ← textual inversion files

For A1111:

stable-diffusion-webui/
  models/
    Stable-diffusion/  ← checkpoint files
    Lora/              ← LoRA files
    VAE/               ← VAE files

LoRA Models (Style Modifiers)

LoRA models are small weight files (50-300 MB) that apply a specific style on top of a base model. Popular uses: specific artistic styles, consistent character faces, product photography styles.

In A1111, add a LoRA by including its trigger keyword in your prompt with a weight: ``. In ComfyUI, use the Load LoRA node and connect it between the model loader and the sampler.

Writing Effective Prompts

Stable Diffusion prompts follow a different structure than conversational AI tools. The model responds to weighted keyword phrases, not natural sentences.

Positive Prompt Structure

A good positive prompt layers descriptors from general to specific:

[subject], [medium/style], [lighting], [composition], [quality keywords]

Example for a portrait:

a young woman with red hair, professional headshot photography, soft studio lighting, shallow depth of field, 85mm lens, sharp focus, 8k resolution

Negative Prompt — What to Exclude

Negative prompts tell the model what to avoid. A standard negative prompt for photorealistic images:

blurry, out of focus, low quality, jpeg artifacts, watermark, signature, text, ugly, deformed, bad anatomy, extra limbs, missing fingers, cropped

Prompt Weighting

Both ComfyUI and A1111 support weight adjustments:

Example: `(photorealistic:1.3), (sharp focus:1.2), woman, garden, [cartoon:0.3]`

Troubleshooting

Frequently Asked Questions