🧠⚡ Claude Code Local

Run Claude Code 100% on-device with local AI on Apple Silicon.
No cloud, no API key, no proxy — an MLX-native server that speaks the Anthropic API.
🥊 Pick your fighter: Hermes 4 14B · Gemma 4 31B · Muse-Glimmer 30B · Qwen 3.8 27B bf16 · Llama 3.3 70B · Qwen 3.5 122B · DeepSeek V4 Flash (1M context via ds4).

🛑 Usage Limit · 🤔 What Is This · 🚀 Quick Start · 🏆 Leaderboard · 🥊 Lineup · 🎮 Modes · 🔒 Privacy · 📊 Benchmarks · 🎤 Voice · 🌐 Browser · 📱 Phone · 🔌 MCP · 🛣️ Roadmap

🛑 Hit your Claude usage limit?

If Claude Code just told you "you've reached your usage limit" and gave you a reset time hours away, that's what this is for. You keep working — same Claude Code, same terminal, same project — except the model answering is running on your own Mac.

curl -fsSL https://raw.githubusercontent.com/nicedreamzapp/claude-code-local/main/install.sh | bash

No API key. No second subscription. No waiting until 3pm. It works on a 16 GB MacBook and gets better the more RAM you have — see what runs on your Mac.

Real session, unedited. Claude Code reads and edits the file — the model answering is Gemma 4 31B on the laptop.

🤔 What Is This?

Your Mac has a powerful GPU built right into the chip. This project uses that GPU to run massive AI models — the same kind that power ChatGPT and Claude — entirely on your computer, and plugs them into Claude Code so the whole coding experience works offline.

No internet, no subscription, nobody sees your code — and it's the full Claude Code experience: edit files, manage projects, drive your browser, or run a hands-free voice session.

The trick: Claude Code speaks the Anthropic API. Local model servers speak the OpenAI API. So everyone bolts a translation proxy in between — and the proxy is slow and fragile. This server speaks Anthropic natively. One process, zero translations:

🎯 That one change — eliminating the proxy — made it 7.5× faster.

🎬 Watch It Run — AirGap AI

A real NDA. Llama 3.3 70B. Wi-Fi physically OFF. lsof running live. Watch a 70-billion-parameter model audit a confidential legal document, on-device, with the receipts on screen.

AirGap is this whole build running as one private workstation — a capability, not a product. Everything you need is in this repo. If your firm needs one built, here's what it looks like.

More local-AI demos on the channel:

🏆 Which model should I run? — The Agent-12 Leaderboard

The question every issue here eventually asks. So we measure it. Agent-12 runs each local model through real agent tasks in a sandboxed working directory, judged by what ends up on the filesystem, never by the model's prose. Temperature 0, fixed caps, fresh sandbox per task, one variable moved per comparison. Tasks, judges, runner and the judge-validation gate are all open at github.com/nicedreamzapp/agent12.

All local rows: Apple M5 Max, 128 GB. The headline: the local champion clears the same hard suite as cloud Claude, and does it faster on wall-clock (125s vs 131s) because there is no network in the loop. Read the launch story: I took down six of my own benchmark videos, then built the leaderboard · 71-second video.

Qwen 3.8 reaches 8/8 hard when given an 8000-token budget, but at 16.9× Qwen3.6's wall-clock (full writeup). Muse-Glimmer 30B and Nemotron Omni are listed on the board in a separate vendor-reported section with Meta's and NVIDIA's own published numbers, credited and linked, until they get a real Agent-12 run.

🥊 The Lineup — Pick Your Fighter

We started with one model. Now we ship a roster — and it's a living lineup: we're builders, this repo is always testing and updating, and new fighters get added the day they drop and benchmarked as we run them. Same MLX server, same Anthropic API — swap one env var and you swap the brain. Plus the ds4 engine for DeepSeek V4 Flash via its own native Metal runtime.

🟣 Qwen 3.8 27B just landed (Aug 20, 2026) — and we run it at full bf16. Alibaba's new 27B dense model (Apache 2.0, native image + video input, 262K context) is the first small model we'd put next to the cloud ones, so we refused to quantize it. The trick that makes bf16 livable is DFlash 2, a speculative-decoding drafter from Inco AI / Z Lab: a 2B draft model proposes a block of tokens, the 27B verifies the block in one pass, and the output is byte-identical to plain decoding. Measured on our M5 Max 128 GB, same prompt, 600 tokens, greedy: 9.7 tok/s plain → 36.5 tok/s with DFlash 2 (3.8×, 4.4 accepted tokens per round). Weights: mlx-community/Qwen3.8-27B-bf16 (54.7 GB) + drafter incoai/Qwen3.8-27B-DFlash2 (3.8 GB), served by mlx-dspark (pip install mlx-dspark, OpenAI-compatible API). Two tips that survived a night of testing: keep the draft block at 5 on Metal (4/6/7/10 were all slower for us and for others), and drop the community Sharp chat template into the model folder — it fixes the stock template's empty-think aborts, defaults reasoning to medium instead of xhigh, and makes the model lead with the answer. Vision works through mlx-vlm (no drafter on that path yet). Claude Code launcher is next on the list.

🧪 Muse-Glimmer just landed (Aug 2026) — Meta's new agentic 30B, abliterated in-house (our first self-abliteration). Decode speed is measured — ~18 tok/s on an M-series Max, 8-bit (a touch quicker than Gemma 4 31B).

👁️ Now with vision via the -MM-bf16 build — watch it read a chart, a neon sign and a blurred car badge. Details in our uploads.

💻 Got a 16 GB MacBook Air? Start with Hermes. setup.sh picks it for you automatically — you don't need 96 GB of RAM to use this.

💡 Fun fact: Qwen wins raw speed because it's an MoE — only 10B of 122B params activate per token. DeepSeek V4 Flash is even bigger (284B) but only ~37B active per token, and it ships with on-disk KV cache so a 25k-token Claude Code system prompt prefills exactly once, ever.

🐳 DeepSeek V4 Flash via ds4

We tested it the day Antirez (the Redis guy) shipped ds4. Local DeepSeek beat cloud Claude on wall-clock time on the same MacBook, same prompt — watch the three-way.

⭐ Our Own MLX Abliterated Uploads

The models in this lineup aren't from generic mirrors — we package and upload our own abliterated MLX builds to HuggingFace so anyone running this repo can pull them with one command. Browse the full set at huggingface.co/divinetribe.

# Llama 3.3 70B — full-precision feel
MLX_MODEL=divinetribe/Llama-3.3-70B-Instruct-abliterated-8bit-mlx \
  bash scripts/start-mlx-server.sh

# Gemma 4 31B — fast daily driver
MLX_MODEL=divinetribe/gemma-4-31b-it-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# Hermes 4 14B — sweet spot for 16/32 GB Macs
MLX_MODEL=divinetribe/Hermes-4-14B-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# Muse-Glimmer 30B — Meta's new agentic model, abliterated in-house
MLX_MODEL=divinetribe/Muse-Glimmer-30B-Abliterated-8bit \
  bash scripts/start-mlx-server.sh

Every public model on huggingface.co/divinetribe, as of Aug 20, 2026. Sizes are the real on-disk totals from the Hub. All are MLX-format; the 👁️ ones take images.

Text models

Vision + text models 👁️

Also on the Hub: yolov8n-oiv7-coreml, the 601-class CoreML detector behind RealTimeAICam.

Abliteration sources: huihui-ai (Llama, Qwen, one of the Gemma 31Bs), null-space (Gemma 4 31B), OpenYourMind (Gemma 4 12B) and Babsie (Hermes). Muse-Glimmer and Nemotron Omni we abliterated ourselves — refusal direction removed across every residual-writing layer (layer 26 on Glimmer), on the freshly-released bf16 weights, with the vision (and for Nemotron, audio) towers kept intact. We believe Glimmer-MM was the first abliterated multimodal model running on Apple MLX. MLX conversion + quantization by us. See what abliteration means.

⚠️ Use it responsibly. "Abliterated" suppresses the model's built-in refusal direction so it doesn't refuse benign-but-edgy requests. It is not a general capability upgrade, and you remain bound by each upstream license (Llama 3.3, Gemma, Hermes/Qwen3, Qwen3.6/VL, Muse-Glimmer, Nemotron).

🎮 The Modes

Four ways to run the lineup. Each one is a double-clickable launcher in launchers/.

💻 What You Need

Also need:

• 🐍 Python 3.12+ (for MLX)
• 🤖 Claude Code (npm install -g @anthropic-ai/claude-code)

🚀 Quick Start (One Command)

curl -fsSL https://raw.githubusercontent.com/nicedreamzapp/claude-code-local/main/install.sh | bash

Or clone it yourself if you'd rather read the script first:

git clone https://github.com/nicedreamzapp/claude-code-local
cd claude-code-local
bash setup.sh

setup.sh auto-detects your RAM, picks a model from the lineup, downloads it, installs the MLX server, and creates a Claude Local.command launcher on your Desktop.

Then double-click Claude Local.command. You're coding locally.

🐛 If the launcher asks you to sign in to a Claude account: your claude CLI is too old. The launchers pass --bare to force local-only API-key auth; older CLIs don't support it. Fix: npm install -g @anthropic-ai/claude-code

🛠️ Note for contributors: setup.sh installs the server as a symlink at ~/.local/mlx-native-server/server.py pointing back at this repo's proxy/server.py. Edit the file in the repo, restart the server, done — one source of truth, no silent drift.

Or do it manually

# 1. Set up the MLX virtualenv
python3.12 -m venv ~/.local/mlx-server
~/.local/mlx-server/bin/pip install mlx-lm

# 2. Pick a fighter and download (one time, ~18-75 GB)
bash scripts/download-and-import.sh gemma   # or 'llama' or 'qwen'

# 3. Start the server
MLX_MODEL=divinetribe/gemma-4-31b-it-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# 4. Launch Claude Code
ANTHROPIC_BASE_URL=http://localhost:4000 \
ANTHROPIC_API_KEY=sk-local \
claude --model claude-sonnet-4-6

🔧 How It Works

┌──────────────────────────────────────────────────┐
│              Your MacBook (M-series)             │
│                                                  │
│  📝 You type ──> 🤖 Claude Code                  │
│                      │                           │
│                      ▼                           │
│                 ⚡ MLX Server (port 4000)        │
│                      │                           │
│                      ▼                           │
│                 🥊 Local model ──> 🖥️  GPU        │
│                 (Gemma·Llama·Qwen)               │
│                      │                           │
│                      ▼                           │
│  📝 Answer <─── ✨ Clean response                │
│                                                  │
│         🔒 Nothing leaves this box. Ever.        │
└──────────────────────────────────────────────────┘

The server (proxy/server.py) is one file, ~1000 lines. It does six things:

1. 📦 Loads the model — Apple's MLX framework, native Metal GPU, unified memory. Handles Gemma's RotatingKVCache quirk automatically.
2. 🔌 Speaks Anthropic API — Claude Code thinks it's talking to Anthropic's cloud. It's not.
3. 🔧 Translates tool use — Three tool-call formats in and out: Gemma 4 native, Llama 3.3 raw JSON, and HuggingFace <tool_call> JSON (Qwen and others). All converted ↔ Anthropic tool_use blocks, with garbled-output recovery for small models.
4. 🧹 Cleans the output — A real-time ThinkingFilter strips <think> blocks token-by-token during generation, then clean_response handles stop markers and reasoning preamble.
5. ⚡ Reuses prompt caches across requests — Claude Code's system prompt doesn't get re-prefilled every turn. Huge speedup for short questions.
6. 🎯 Code mode — auto-detects Claude Code coding sessions, swaps the ~10K-token harness prompt for a slim ~150-token one, and strips verbose tool descriptions to name + parameter types. A 28× prompt reduction that cuts prefill from ~60 s to ~2 s on Gemma 4 31B.

🔒 Privacy + How the Data Flows

This is the part we're proudest of. Your code never leaves your Mac. Not for a model call. Not for telemetry. Not for "anonymous analytics". Not ever.

   ┌─────────────────────────────────────────────────────────────┐
   │                    🖥️  YOUR MACBOOK                          │
   │                                                             │
   │   📝 Your code ──> 🤖 Claude Code ──> ⚡ MLX Server          │
   │                     (localhost:4000)      │                 │
   │                                           ▼                 │
   │                    🧠 Local model ──> 🖥️  Apple GPU          │
   │                                                             │
   │             🚫 ZERO outbound network calls                  │
   │             🚫 ZERO telemetry                               │
   │             🚫 ZERO phone-home                              │
   └─────────────────────────────────────────────────────────────┘
                   │
                   ✗  ←  Nothing from our code crosses this line.
                   │
   ┌─────────────────────────────────────────────────────────────┐
   │                    ☁️  THE INTERNET                          │
   │                  (your code never goes here)                 │
   └─────────────────────────────────────────────────────────────┘

🔍 What We Audited (Every Component)

✅ Verified offline. Claude Code's own binary previously reached out to api.anthropic.com on startup for telemetry, statsig feature flags, marketplace auto-install, and the autoupdater. The launchers plug all four channels via documented Anthropic env vars (thanks @tadrianonet, PR #32):

CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
DISABLE_AUTOUPDATER=1
CLAUDE_CODE_DISABLE_OFFICIAL_MARKETPLACE_AUTOINSTALL=1
CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1

Verify it yourself: run lsof -p $(pgrep -f claude) during a session — you'll see only localhost:4000. Run lsof -i -P while the server is up — nothing leaves your Mac.

⚠️ We removed LiteLLM after supply-chain attack concerns. Every dependency was re-audited from scratch. If a package had unexplained network calls, it didn't ship.

📊 Benchmarks

⚡ Speed Comparison

⚡ Gen 4 — The Native Engine (added Aug 7, 2026)

New, optional, and additive. Every Claude Code launcher in this repo works exactly as it did yesterday. This is a second way in, for people who want the fastest possible turnaround out of the same local models.

Why we built it. Claude Code is a great harness, but it was designed for cloud models: its system prompt is tens of thousands of tokens, and parts of the prompt change every turn. A local model pays for that twice — once in prefill time, and again because a mutating prompt head defeats KV-cache reuse entirely. The native engine (agent/agent.py, ~900 lines, zero dependencies beyond mlx-lm) flips the design: a ~550-token system prompt that never changes, the same tools (Bash/Read/Write/Edit/Glob/Grep), and a KV cache that is trimmed to the shared prefix each turn so only the new tokens are ever prefilled.

What we found on the way (useful even if you never run this). While benchmarking we discovered that Gemma-family models hand their sliding-window layers a RotatingKVCache capped at the window (1024 tokens on Gemma 4). The moment a conversation outgrows the window, those caches report untrimmable and prompt-cache reuse silently dies — every turn re-prefills the entire transcript, exactly in the long sessions where caching matters most. The engine swaps in plain KVCache on every layer: the sliding-window attention mask still enforces the window, so outputs are byte-identical (we verified with a greedy A/B comparison), but the cache stays trimmable forever. AGENT_ROLLING_KV=1 restores stock behavior if you want the lower memory ceiling instead.

Measured (Gemma 4 31B 4-bit, 4.5k-token conversation, Apple Silicon):

Roughly 18× faster turn starts on long sessions, with generation steady at 26–28 tok/s. Reproduce it yourself: python3 bench/agent_bench.py.

Terminal UX: a framed input area with a live context-percentage meter in the top rule, a thinking/running spinner so silence never means "hung", and readline history across sessions.

Run it: double-click a (Native Engine) launcher, or:

AGENT_MODEL=lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit \
AGENT_DIALECT=native \
python3 agent/agent.py

AGENT_DIALECT=native uses the model's own tool-call template (Qwen3-Coder's XML with raw-text values); prompted teaches the same XML dialect in the system prompt for models whose native format re-introduces JSON escaping (Gemma). AGENT_BACKEND=http points the same agent at any local Anthropic-protocol server, including this repo's proxy.

This builds directly on the prompt-cache trim work contributed in #46 — that fix is what made the deeper rotating-cache problem visible. Thank you.

☁️ vs Cloud APIs

Qwen 3.5 numbers measured on an M5 Max 128 GB — full runs, including Qwen 3.8 at bf16, in BENCHMARKS.md.

🔧 Tool-Call Reliability

Local models don't format tool calls perfectly. They want to call a tool but mix XML and JSON syntax — Claude Code sees no valid tool call, re-prompts, and the model garbles it the same way again. The result: infinite loops where the AI says "let me do that" but never does anything.

We fixed this with 4 changes to server.py:

🧪 Results: 98/98 tests passed across 7 consecutive runs. Zero failures. The multi-step scenario that used to trigger infinite loops — create 12 month folders, delete all but September, verify — now passes every time. Run it yourself:

python3 scripts/test_mlx_server.py

⚙️ Tuning

📚 More

Everything above gets you running. These live in docs/ so this page stays short:

🧩 The Local-First Stack

claude-code-local is the brain. It pairs with sibling repos — each stands alone, together they take Claude Code off the keyboard and off the screen:

🔬 More Local-AI Work

Not part of this stack — separate projects, same rule: the model runs on your machine, not on somebody's server.

📱 There are four apps on the App Store and Google Play too — see them here.

🛣️ What's Next

We ship fast and in public. If any of these excite you, hit Watch to get the release ping.

• 🟣 Qwen 3.8 27B launcher — wire the bf16 + DFlash 2 server into Claude Local.command and the tool-call translator (today it's served by mlx-dspark on its own port)
• 🟡 Full Qwen 3.5 122B benchmark suite — reliability, tool-call pass rate, long-context behavior vs Gemma
• 🟡 Fully-local Whisper fallback — alternative to the Apple SFSpeechRecognizer path for older Macs and non-English voices
• 🟡 One-click DMG installer — no terminal needed
• 🟡 MLX_MODEL=<hf-url> — point at any HuggingFace repo and auto-register a new fighter
• 🟡 More fighters — open to PRs adding launchers for DeepSeek, Mistral, Phi, anything MLX-compatible

💡 Want something that's not on this list? Open an issue → Every serious request gets read and usually replied to within 24h.

🤝 Contributing

Ideas, bug reports, a new launcher for a model I don't run, a better code-mode prompt — open an issue or a PR, I read them all. Especially interested in: folks on older Apple Silicon (M1/M2, 16–36 GB) who know which models actually fit; anyone stress-testing the voice loop on different hardware or accents; TTS recipes beyond Pocket TTS (Piper, MLX-TTS, Kyutai Moshi); and edge cases I'll never hit on an M5 Max with 128 GB.

🙏 Credits

🧑‍🔧 Contributors

Every one of these landed on hardware I don't own, on a bug I hadn't hit. Thank you.

Tested on Apple M5 Max with 128 GB unified memory.

Built by Matt Macosko in Arcata, CA — one person, no team, no investors; the MLX server, the leaderboard, the abliterated uploads, the voice loop and the phone bridge are all his. Open to work on local-AI and Apple Silicon inference — matt@ineedhemp.com. Part of Nice Dreamz LLC. More open-source at nicedreamzwholesale.com/software · demos at youtube.com/@nicedreamzapps.

📜 MIT License — Use it however you want.

💬 Builders hang out on Discord — share what you're building, swap MLX tips.

⭐ Star this repo if it helped you! ⭐

Upstream projects this is built on are listed in docs/CREDITS.md.