Run LLMs Locally
Complete guide to running AI models on your own hardware. Private, fast, and free after setup.
Hardware Requirements
| Model Size | Min RAM | Recommended RAM | GPU VRAM | Example Models |
|---|---|---|---|---|
| 7B | 8GB | 16GB | 6GB | Llama 3.2, Mistral 7B, Phi-3 |
| 13B | 16GB | 32GB | 10GB | CodeLlama 13B, Llama 2 13B |
| 30-34B | 32GB | 64GB | 24GB | CodeLlama 34B, Mixtral 8x7B |
| 70B+ | 64GB | 128GB | 48GB+ | Llama 3 70B, DeepSeek |
Which Model Should You Use?
Coding Assistant
General Chat
Writing & Docs
Image Understanding
Step-by-Step Setup
macOS (Apple Silicon)
Install Homebrew
Package manager for macOS
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Install Ollama
The easiest way to run LLMs locally on Mac
brew install ollama
Start Ollama Service
Runs in the background, manages models
ollama serve
Download a Model
Pulls Llama 3.2 (4.7GB). Other options: mistral, codellama, phi3
ollama pull llama3.2
Chat with Your Model
Start an interactive chat session
ollama run llama3.2
💡 Tip: Use 'ollama list' to see downloaded models
Cost Comparison: Local vs API
| Method | Setup Cost | Monthly | Per 1M Tokens | Latency | Privacy |
|---|---|---|---|---|---|
| Local (7B) | $0 | $0* | $0 | 20-100ms | ✓ Full |
| Local (70B) | $1,500+** | $50-100 | $0 | 100-500ms | ✓ Full |
| OpenAI GPT-4 | $0 | Usage | $30-60 | 500-2000ms | ✗ Cloud |
| Claude 3.5 | $0 | Usage | $15-75 | 500-2000ms | ✗ Cloud |
| GPT-4 API | $0 | Usage | $30+ | 1-3s | ✗ Cloud |
* Electricity cost only (~$5-10/month for heavy use)
** High-end GPU (RTX 4090 or multiple) for 70B models
Pro Tips
Start with 7B models
They run on almost any hardware and are surprisingly capable
Use quantized models
Q4_K_M offers best balance of speed and quality
Monitor GPU memory
Use nvidia-smi or Activity Monitor to check usage
Try different models
Each has strengths - Mistral for reasoning, CodeLlama for code
Use the API
Ollama serves OpenAI-compatible API at localhost:11434
Join the community
r/LocalLLaMA and AGI House Discord for tips
Download the Complete PDF Guide
Includes troubleshooting tips, model comparison charts, and advanced configuration.
No spam, ever. Unsubscribe anytime.