How to Run Mistral Medium 3.5 Locally with Ollama (2026 Guide)

What Is Mistral Medium 3.5?

Mistral Medium 3.5 is Mistral AI's first "merged" model, meaning a single set of weights covers instruction-following, reasoning, and coding instead of splitting them across separate models. Mistral AI released it on April 29, 2026, explicitly positioning it to replace Mistral Medium 3.1, Magistral, and Devstral 2 in its production lineup.

Unlike most of the other recent flagship releases covered on this site (Kimi K2.6, GLM 5.2, MiniMax M3), Mistral Medium 3.5 is dense rather than mixture-of-experts. Every one of its 128 billion parameters activates on every forward pass, which is part of why even the smallest quantized tag needs 80GB just for the weights.

On Mistral's published benchmarks, it scores 77.6% on SWE-Bench Verified, ahead of Mistral's earlier coding-focused models, and over 90% on Mistral's internal tool-use benchmark for enterprise agent tasks. It supports 40-plus languages, function calling, JSON output, and a configurable reasoning effort setting, plus native vision input through an encoder trained to handle variable image sizes and aspect ratios, useful for documents, diagrams, and UI screenshots. It ships under a Modified MIT license: free for commercial and noncommercial use, with exceptions for companies above a certain revenue threshold.

Install Ollama and Choose Your Quantization Tag

Step 1: Match a tag to your hardware

If you are not sure, start with the default q4_K_M tag. It is the only realistic option on a single 80GB-class GPU and is what the rest of this guide uses.

Step 2: Install Ollama

Linux:

curl -fsSL https://ollama.com/install.sh | sh

macOS:

brew install ollama

Windows: download the installer from ollama.com and run it. WSL2 users can run the Linux command above inside their distro.

Verify the install:

ollama --version

ollama version is 0.12.3

Step 3: Pull Mistral Medium 3.5

ollama pull mistral-medium-3.5

pulling manifest
pulling 9a3f21c8... 100% ▕████████████████▏  80 GB
pulling 4b7e8d12... 100% ▕████████████████▏  12 KB
verifying sha256 digest
writing manifest
success

Step 4: Verify the pull

ollama list

NAME                          ID              SIZE      MODIFIED
mistral-medium-3.5:latest     a1b2c3d4e5f6    80 GB     2 minutes ago

If you want a higher-precision tag instead, pull it explicitly: `ollama pull mistral-medium-3.5:128b-q8_0` or `ollama pull mistral-medium-3.5:128b-bf16`.

Run Your First Inference

Step 5: Start a chat session

ollama run mistral-medium-3.5

>>> Write a Python function that deduplicates a list of dictionaries by a given key, and explain the time complexity.

Mistral Medium 3.5 walks through the dictionary keys, builds the function using a seen-set to track which key values it has already encountered, and returns the deduplicated list. Expect an explanation that the approach runs in O(n) time, since each item is checked once against the seen-set rather than compared against every other item.

Step 6: Test vision input

>>> What's wrong with this UI screenshot? ./screenshot.png

Ollama detects the file path in your input and attaches it as an image automatically. This works for any vision-capable model in Ollama, not just Mistral Medium 3.5. The model reads the screenshot through its native vision encoder and responds based on what it sees, no separate upload step needed.

Step 7: Control reasoning effort

For complex, multi-step problems, ask for deeper reasoning through Ollama's `think` parameter, the same mechanism used for other reasoning models like DeepSeek R1 and Qwen3:

curl http://localhost:11434/api/chat -d '{
  "model": "mistral-medium-3.5",
  "messages": [{"role": "user", "content": "Debug this race condition and explain the fix"}],
  "think": "high"
}'

Set `think` to `"low"` for short, low-latency answers on simple prompts, and `"high"` when you need deeper multi-step reasoning for debugging, math, or planning tasks.

Set Up API Access

Ollama exposes Mistral Medium 3.5 through its standard REST API on `localhost:11434`. No separate account or API key is needed for local use.

curl http://localhost:11434/api/generate -d '{
  "model": "mistral-medium-3.5",
  "prompt": "Summarize the tradeoffs between dense and mixture-of-experts model architectures",
  "stream": false
}'

Python example:

import requests

response = requests.post('http://localhost:11434/api/chat', json={
    'model': 'mistral-medium-3.5',
    'messages': [{'role': 'user', 'content': 'Write a SQL query to find duplicate rows by email'}],
    'stream': False
})

print(response.json()['message']['content'])

For function calling and JSON-mode output, set the `format` field to `json` in the request body or pass a `tools` array following the OpenAI-compatible schema, the same pattern covered in the Ollama with Python guide. If you are wiring Mistral Medium 3.5 into an agent framework, point its OpenAI-compatible base URL at `http://localhost:11434/v1`, the same setup used in the Hermes agent guide.

Troubleshooting

Frequently Asked Questions