How to Run an LLM Locally on Mac with Ollama

TL;DR: Ollama lets you run large language models locally on your Mac with a single command. Install it, pull a model like Llama 3 or Gemma, and start chatting — no cloud, no API keys, no data leaving your machine. Apple Silicon Macs (M1–M5) with 8 GB+ RAM can run 7B–13B parameter models comfortably.

Running an LLM locally on your Mac gives you complete privacy, zero API costs, and instant responses without internet dependency. Ollama is the easiest way to do it: it handles model downloads, quantization, memory management, and GPU acceleration automatically. This guide walks you through the full setup — from install to your first conversation — on Apple Silicon.

Prerequisites and Apple Silicon Requirements

Before installing Ollama, make sure your Mac meets these requirements:

• Chip: Apple Silicon (M1, M2, M3, M4, or M5). Intel Macs work but run significantly slower without Metal GPU acceleration.
• macOS: 12.3 (Monterey) or later.
• RAM: 8 GB minimum. 16 GB recommended for 7B models. 32 GB+ required for 13B+ models.
• Storage: At least 10 GB free. Individual models range from 2 GB (small) to 40 GB+ (large).

Apple Silicon's Unified Memory Architecture is uniquely suited for local LLMs because both CPU and GPU share the same memory pool. A MacBook Air M1 with 16 GB RAM can run a 7B model at roughly 30 tokens per second — fast enough for real-time conversation.

Installing Ollama

There are three ways to install Ollama on macOS:

Method 1: Direct Download (Recommended for Beginners)

1. Visit ollama.com and click Download for macOS
2. Open the downloaded .dmg file
3. Drag Ollama to your Applications folder
4. Launch Ollama from Applications — it runs as a menu bar app

Method 2: Homebrew

brew install ollama

Method 3: Command Line

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation:

ollama --version
# Output: ollama version 0.9.x

Ollama runs as a background service. Once installed, the ollama CLI is available in your terminal and the API server starts on http://localhost:11434.

Pulling and Running Your First Model

Running a model is a single command. Ollama downloads it automatically on first use:

# Pull and start Llama 3.3 (8B parameters, ~4.7 GB download)
ollama run llama3.3

You will see a download progress bar, then an interactive chat prompt:

>>> What is retrieval-augmented generation?
Retrieval-Augmented Generation (RAG) is a technique that combines...

Type your messages and press Enter. Use /bye or Ctrl+D to exit the chat.

Essential CLI Commands

# List all installed models
ollama list

# Pull a model without starting chat
ollama pull gemma3

# Remove a model to free disk space
ollama rm llama3.3

# Show model details (parameters, quantization, size)
ollama show llama3.3

# Run with a system prompt
ollama run llama3.3 --system "You are a Python expert. Answer concisely."

Best Small Models for Mac (2026)

Choosing the right model depends on your RAM and use case. Here are the best options for Apple Silicon Macs:

# Install the best starter model
ollama pull llama3.3

# For a faster, lighter option
ollama pull gemma3:4b

# For deep reasoning tasks
ollama pull deepseek-r1:8b

Using the Ollama API

Ollama exposes a REST API on localhost:11434 that is compatible with the OpenAI API format. This makes it easy to integrate with any application:

# Generate a response via curl
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.3",
  "prompt": "Explain embeddings in 2 sentences.",
  "stream": false
}'

Using Ollama with Python

The official ollama Python library provides a clean interface:

import ollama

# Simple generation
response = ollama.chat(
    model='llama3.3',
    messages=[
        {'role': 'user', 'content': 'What is a vector database?'}
    ]
)
print(response['message']['content'])

For streaming responses (essential for chat UIs):

import ollama

stream = ollama.chat(
    model='llama3.3',
    messages=[{'role': 'user', 'content': 'Write a Python function to sort a list.'}],
    stream=True
)

for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

Install the library with:

pip install ollama

Because Ollama uses the OpenAI-compatible API format, you can also use it as a drop-in replacement with LangChain, LlamaIndex, or any OpenAI SDK — just point the base URL to http://localhost:11434/v1.

RAM and Performance Expectations

Apple Silicon performance depends on your chip and available memory:

Key rule: The model must fit entirely in RAM. If it does not, macOS will swap to disk and performance drops to unusable levels. Check model size with ollama show <model> before running.

Tip: Close memory-heavy apps (Chrome, Docker) before running large models. Use Activity Monitor to check available memory.

Advanced Features (2025–2026)

Ollama has evolved rapidly. Key features to know about:

• Native Thinking Support (v0.9+): Models like DeepSeek-R1 now separate their internal reasoning from the final response. The API returns distinct thinking and content fields for cleaner integration.
• Tool Calling with Streaming: Models can call external functions while streaming responses, enabling real-time agentic workflows.
• Web Search API: A built-in web search capability lets models ground answers in live web results.
• Multimodal Engine: Ollama now supports vision models natively — send images alongside text prompts for analysis.
• Official GUI: As of July 2025, Ollama has a built-in graphical interface for users who prefer a ChatGPT-like experience over the terminal.

Common Errors and Fixes

"Error: model not found" — The model name is case-sensitive. Use ollama list to see available models, or check ollama.com/library for the exact name.

Extremely slow generation — The model is too large for your RAM. Try a smaller variant: ollama run llama3.3:8b instead of the 70B version. Close other applications to free memory.

"bind: address already in use" — Another instance of Ollama is already running. Stop it with pkill ollama and restart.

GPU not being used — Verify you are on Apple Silicon with uname -m (should output arm64). Ollama automatically uses Metal GPU acceleration on Apple Silicon — no configuration needed.

Model downloaded but not responding — Restart the Ollama service: quit the menu bar app and relaunch it, or run ollama serve in a new terminal window.

FAQ

Q: Is Ollama free? A: Yes. Ollama is completely free and open-source. You pay nothing — not for the software, not for the models, and not for usage. All inference runs on your local hardware.

Q: Can I use Ollama with LangChain or LlamaIndex? A: Yes. Ollama exposes an OpenAI-compatible API at localhost:11434/v1. Both LangChain and LlamaIndex have built-in Ollama integrations. You can also use the standard OpenAI Python SDK by changing the base_url.

Q: How does Ollama compare to LM Studio? A: Ollama is CLI-first and optimized for developers who want API access and scriptability. LM Studio provides a polished GUI for downloading and chatting with models. Both run models locally on Apple Silicon. Ollama is the better choice for integration into code and pipelines.

Q: What is the largest model I can run? A: It depends on your RAM. As a rule of thumb, you need roughly 1 GB of RAM per 1 billion parameters (at 4-bit quantization). A 64 GB Mac can run a 70B model. A 16 GB Mac maxes out around 13B.

Q: Can I fine-tune models with Ollama? A: Ollama is for inference, not training. For fine-tuning, see our LoRA Fine-Tuning Tutorial. You can then import your fine-tuned model into Ollama using a Modelfile.

Next Steps

Now that you have a local LLM running, explore these related guides:

• LangChain RAG Tutorial in Python — Build a retrieval-augmented generation app using your local Ollama model
• ChromaDB Tutorial for Python — Set up a local vector database to pair with your LLM
• Introduction to RAG — Understand the RAG pipeline from end to end
• Building AI Agents with LangChain — Use your local model to power an autonomous agent
• Start Here — Our complete learning roadmap