How to Run Gemma 4 on Ollama: Complete Setup Guide (2026)

What Gemma 4 Is and Which Size to Run

Gemma 4 is Google DeepMind's fourth-generation open model family, released on April 2, 2026. It comes in five sizes built around two goals: E2B and E4B target edge devices with a 128K context window, while the 12B, 26B A4B, and 31B models target desktops and workstations with a 256K context window.

The biggest change from Gemma 3 is the license. Gemma 1 through 3 shipped under a custom Gemma Terms of Use that required attribution and banned using the model to train competing models through distillation. Gemma 4 ships under a standard Apache 2.0 license with neither restriction, so you can fine-tune it on private data and sell the result without disclosing weights or training data.

The 26B variant, labeled 26B A4B, uses a mixture-of-experts architecture. Only about 4 billion parameters activate per token, so it generates text close to 4B-model speed, but Ollama still loads all 26 billion parameters into memory, which means the RAM requirement matches a 26B model rather than a 4B one. Every Gemma 4 size also ships with a dedicated draft model for speculative decoding, which Google says speeds up inference with no quality loss.

On Ollama, the `gemma4` library holds 47 tags across these five sizes, plus Apple Silicon MLX builds and one cloud-hosted tag, and has logged more than 13.5 million pulls as of June 2026.

For comparison, Gemma 3 topped out at a 27B model with a 128K context window (32K on its smallest 1B model) under the older Gemma license. Gemma 4's 31B model adds 4 billion more parameters, doubles the context on the larger sizes to 256K, and removes the distillation restriction entirely.

Install Ollama and Run Your First Gemma 4 Prompt

Installing Ollama and getting Gemma 4 running takes about five minutes on a normal connection, most of it spent on the model download.

Step 1: Install Ollama

# Linux and macOS, one-command installer
curl -fsSL https://ollama.com/install.sh | sh

On Windows, download the installer from ollama.com/download, or use winget:

winget install Ollama.Ollama

Confirm the installed version supports Gemma 4 (Ollama 0.6.x or later):

ollama --version
# Expected: ollama version 0.6.x or higher

Step 2: Pull and Run gemma4

The plain `gemma4` tag points at the E4B variant (9.6 GB), a reasonable default for a machine with 16 GB of RAM:

ollama run gemma4

Expected output on first run:

pulling manifest
pulling 7a3c9e21... 100% ▕████████████████▏ 9.6 GB
pulling tokenizer...   100% ▕████████████████▏ 4.2 MB
success
>>> Send a message (/? for help)

Step 3: Send a Test Prompt

>>> Explain the difference between a mixture-of-experts model and a dense model in two sentences.

Gemma 4 streams its response directly in the terminal once the model finishes loading into memory, which takes a few seconds on the first load and is near-instant on repeat runs.

Step 4: Pull a Different Size

If E4B feels slow, or you have more RAM to spare, pull a different tag from the table in the previous section:

# Smaller, for 4-8 GB RAM machines
ollama pull gemma4:e2b

# Larger, for 32 GB+ RAM workstations
ollama pull gemma4:31b

Switch between them at any time with `ollama run gemma4:e2b` or `ollama run gemma4:31b`. Ollama keeps every pulled model on disk, so you can compare sizes without re-downloading.

Send Images to Gemma 4 (Multimodal Input)

Every standard Gemma 4 tag (e2b, e4b, 12b, 26b, 31b) accepts image input alongside text. The `-mlx` and `-cloud` variants are text-only as of June 2026.

Attach an Image in the Terminal

In an interactive `ollama run` session, include the image path in your message:

>>> What's happening in this chart? /home/user/Downloads/sales-chart.png

Ollama detects the file path, loads the image, and sends both the image and your text to the model in a single request. This works for screenshots, photos, and diagrams. Google's documentation specifically highlights OCR and chart understanding as strengths of the Gemma 4 family.

Send an Image via the API

For scripted use, base64-encode the image and include it in the `images` array of an `/api/chat` request:

IMAGE_B64=$(base64 -w0 /home/user/Downloads/sales-chart.png)

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:e4b",
  "messages": [
    {
      "role": "user",
      "content": "Summarize the trend in this chart in one sentence.",
      "images": ["'"$IMAGE_B64"'"]
    }
  ],
  "stream": false
}'

Expected output (truncated):

{
  "model": "gemma4:e4b",
  "message": {
    "role": "assistant",
    "content": "The chart shows a steady upward trend in sales from January through June, with a sharp jump in May."
  },
  "done": true
}

Audio Input on E2B and E4B

The E2B and E4B variants additionally accept audio input for speech recognition tasks, sent the same way as images, as a base64-encoded clip in the request. The 12B, 26B, and 31B models do not support audio as of this writing.

Configure Context Length and Use the API

Gemma 4's default context windows (128K for E2B/E4B, 256K for 12B/26B/31B) cover most use cases, but a Modelfile lets you adjust the active context and system prompt.

Create a Custom Modelfile

FROM gemma4:e4b
PARAMETER num_ctx 32768
SYSTEM "You are a concise technical assistant. Answer in plain text without markdown formatting."

Build and run it:

ollama create my-gemma4 -f Modelfile
ollama run my-gemma4

Lowering `num_ctx` below the model's maximum reduces KV cache memory overhead, which matters most on the 26B and 31B variants, where the base model already needs 14-18 GB of RAM before context is added.

OpenAI-Compatible Endpoint

Ollama exposes Gemma 4 through the same OpenAI-compatible API used by Hermes Agent and other agent frameworks, at `http://localhost:11434/v1`:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

response = client.chat.completions.create(
    model="gemma4:12b",
    messages=[{"role": "user", "content": "Write a regex that matches US ZIP codes."}],
)
print(response.choices[0].message.content)

Run Gemma 4 with Open WebUI

For a chat interface instead of the terminal, Open WebUI detects every locally pulled Ollama model automatically, including all `gemma4` tags, with no extra configuration needed.

Troubleshooting

Frequently Asked Questions