ccLoad

AI API gateway for Claude Code, Codex, Gemini, and OpenAI.

English | 简体中文

Smart routing | Automatic failover | Model-aware cooldown | Multi-URL scheduling | Protocol transforms | Live monitoring | Cost control

ccLoad removes the operational mess of running multiple AI API upstreams. It keeps Claude Code, Codex, Gemini, and OpenAI-compatible clients on one stable gateway, then handles upstream selection, failover, cooldown, protocol conversion, request visibility, and cost limits in the service instead of in every client script.

🤖 Built with Codex and GPT-5.6

During OpenAI Build Week, Codex powered by GPT-5.6 was the primary engineering agent used to:

• Trace request routing, failover, cooldown, protocol conversion, and dashboard flows across the Go backend and embedded web UI.

• Implement and review model-scoped cooldown handling for upstream 5xx, key-level 429, model-unavailable 404, and explicit model-retirement 410 failures without unnecessarily cooling an entire channel.

• Refine the model-status and call-statistics UI, update the English and Chinese documentation, and verify the result with focused Go tests, builds, and browser walkthroughs.

• Prepare the reproducible demo and Devpost submission while keeping architecture, security, and final-review decisions under human control.

GPT-5.6 is also integrated into the product itself: ccLoad exposes GPT-5.6 through OpenAI-compatible and Codex Responses endpoints, includes Sol, Terra, and Luna model presets, calculates their standard, priority, flex, cached-token, and long-context costs, and applies routing and model-scoped cooldown decisions to them like any other configured upstream model.

The repository's AGENTS.md and CLAUDE.md provide persistent engineering constraints so Codex works against the same KISS-first review and testing rules in every session.

🎯 What ccLoad Solves

Common failure modes when you run several AI API channels:

• Manual channel switching: Different keys, validity windows, quotas, and upstream URLs quickly become hard to manage.

• Rate limits and upstream failures: 429, 502, 504, expired keys, and overloaded providers should not stop the client workflow.

• Opaque request status: Without live request visibility, long streaming requests become guesswork.

• HTTP 200 with error content: Some upstreams return a successful HTTP status while the response body is an actual error.

• Cost drift: Shared gateways need per-channel and per-token limits, not spreadsheet accounting after the bill arrives.

ccLoad handles those cases with:

• Smart routing: High-priority channels are selected first; channels at the same priority use smooth weighted round-robin.

• Automatic failover: Failed keys, models, channels, and URLs are skipped according to the classified error scope.

• Model-aware cooldown: Structured model_cooldown responses, upstream HTTP 5xx failures, key-level 429 rate limits, model-unavailable 404 errors, and explicit model-retirement 410 errors all cool only the actual upstream model first; other models on the same channel remain available. The channel is promoted to cooldown only after every configured model or every enabled key is cooling.

• Multi-URL scheduling: A single channel can use multiple upstream URLs, weighted by observed latency and health.

• Per-URL protocol routing: Each URL can declare the upstream wire protocols it accepts. Explicit declarations route directly; an empty declaration tries the client protocol first and caches the working fallback.

• Responses WebSocket bridging: Authenticated Codex clients can keep a downstream WebSocket while each candidate uses native Codex WebSocket or the existing HTTP/SSE transport.

• Live monitoring: Active requests, logs, token usage, TTFB, cost, and upstream details are visible in the web dashboard.

• Soft-error detection: HTTP 200 responses that are actually errors trigger the same failover path as regular upstream failures. Common cases include:

• JSON responses containing {"error": {...}} structure

• Responses with type field set to "error"

• Explicit rate limits in SSE error events (rate_limit_exceeded / too_many_requests) are handled as 429

• Plain text messages like "当前模型负载过高" / "Current model load too high" (load warnings)

✨ Key Features

• 🚀 High-Performance Architecture - Gin framework, 1000+ concurrent connections, high-performance caching

• 🧮 Local Token Counting - API-compliant local token estimation, <5ms response, 93%+ accuracy, supports large-scale tool scenarios

• 🎯 Smart Error Classification - Distinguishes Key/Model/Channel/Client errors, soft error detection (200 masquerading as error), SSE rate-limit errors as 429, 1308 quota handling

• 🔀 Smart Routing - Priority + smooth weighted round-robin channel selection, pre-filters cooled channels, multi-key load balancing, health-based dynamic sorting (confidence factor prevents small sample over-penalization)

• 🛡️ Failover - Key, model, and channel failures share one exponential-backoff policy; explicit upstream reset deadlines take priority, and model-scoped failures switch channels without cooling the whole channel

• 🔒 Race-Safe - Key selector race condition protection, startup config validation, automatic resource cleanup

• 📊 Real-time Monitoring - Built-in trend analysis, logging, and stats dashboard, Token usage stats with time range selection and per-token classification, runtime status panel with process metrics (CPU, RSS, GC)

• 🎯 Transparent Proxy - Supports Claude Code, Codex, Gemini, and OpenAI compatible APIs with smart auth detection

• 🔑 OAuth Channels - Codex (ChatGPT), Anthropic (Claude), Antigravity, and xAI OAuth credentials with automatic refresh where supported; Codex personal access token (PAT) authorization; Z.ai Coding Plan (ZCode) browser authorization or API-key import; and Cursor user API-key import, with batch quota refresh, invalid-credential cleanup, and auto-disable for permanently rejected credentials

• 📅 OAuth Quota Cost Tracking - Per-credential weekly/monthly standard-cost accumulation aligned to upstream quota windows, plus manual Codex quota reset when a reset credit is available

• 🔌 Responses WebSocket - Downstream Codex WebSocket sessions bridge to native Codex WebSocket or HTTP/SSE candidates with transcript-aware failover

• 📦 Simple Deployment - Embedded SQLite; the Cursor SDK Bridge is managed automatically when needed

• 🔒 Secure Authentication - Token-based admin interface and API access control

• 🏷️ Build Tags - GOTAGS support, high-performance JSON library enabled by default

• 🐳 Docker Support - Multi-arch images (amd64/arm64), automated CI/CD

• ☁️ Cloud Native - Container deployment support, GitHub Actions auto-build

• 🤗 Hugging Face - One-click deployment to Hugging Face Spaces, free hosting

• 💰 Cost Limits - Per-channel daily cost limits, per-token cost limits

• 🚦 Channel RPM Limits - Per-channel rolling 60-second request caps, 0=unlimited

• 🚧 Channel Concurrency Limits - Per-channel in-flight request caps, 0=unlimited

• 🕒 Channel Time Windows - Optional HH:MM availability window per channel (server local time, cross-midnight supported); channels outside their window are fully excluded from routing

• 🔐 Token Restrictions - Per-token cost limits, model restrictions, channel allowlist/denylist, and concurrency caps for fine-grained access control

• ⏱️ TTFB Monitoring - Streaming request first byte time tracking for upstream latency diagnosis

• 🌐 Multi-URL Load Balancing - Multiple URLs per channel with latency-weighted random selection

• 🧭 Per-Channel Proxy - Route a channel's upstream traffic through an http/https/socks5/socks5h proxy with isolated connection pools

• 💵 service_tier Pricing - OpenAI priority/flex/default tier multipliers for accurate cost accounting

• 🖼️ Image Tool Billing - Responses image_generation/gpt-image-2 cost accounting

• 📉 Tiered Pricing - GPT-5.4/Qwen-Plus/Gemini long-context step pricing, auto-applies lower rate at token thresholds

• 🔄 Per-URL Protocol Routing - Explicit Anthropic/OpenAI/Codex/Gemini capability per URL, with native-first automatic detection when left empty

• 💬 Conversational Model Testing - Channel/model/chat testing modes with image upload, reasoning level, built-in search, and chat export

• 🎨 Image Generation Testing - Dedicated tab that renders generated images through either the Images API or Chat Completions, with size/quality/background/output-format controls

• 🔍 Debug Logs - Upstream request/response raw data capture with sensitive header masking, essential for troubleshooting

• 🕐 Scheduled Checks - Background periodic channel availability probing, auto-detect failed channels

• 🔄 Release Channels - Stable updates by default, with an opt-in preview channel; check interval is configurable from the admin settings page

• 🧩 Custom Request Rules - Per-channel HTTP header & JSON body rewriting (remove/override/append), with auth header protection, CRLF guard, and capacity caps

• 🎛️ Log Column Customization - Show/hide table columns per preference, settings persist in bro