ComfyUI-Agent-Kit

Local-first ComfyUI for every AI coding agent (Claude Code, Codex, Gemini CLI, Qwen Code).
Your GPU, your models, no cloud, no account.

By AI VFX NEWS.

Make Claude Code, Codex, Gemini CLI, or Qwen Code drive ComfyUI at full power on your own machine - generate images, video, and audio, build and run workflows, pick the model variant that fits your hardware, and show the graph live in your own ComfyUI canvas. No hosted service, no per-generation billing: one installer wires the same stack into every agent you run, then you hand the whole setup to someone else with one command.

This is the portable, machine-independent, multi-agent version of a working ComfyUI setup. One shared core (the knowledge + the MCP driver) plus a thin adapter per agent. Clone it, run the installer, and each of your agents gets the same stack, wired to your hardware. GLM (z.ai) run through Claude Code is covered by the claude adapter. See docs/AGENTS.md for how each agent connects.

Local-first, for experts and everyday users alike. It scales from one-command image / video generation to a professional VFX color pipeline: v2 ships ComfyUI-OCIO (nine Nuke-style OpenColorIO nodes - Read a sequence, grade in ACES, write ProRes, preview on an on-node player, fully color-managed) and the field guide to building your own custom nodes.

Local-first by design. Prefer the cloud? The official Comfy Cloud MCP runs your workflows on Comfy's GPUs, no local setup. This kit is the local-first counterpart: everything runs on hardware you control, with no account and no per-generation cost, the model picker sizes each job to your VRAM, and it serves four agents, not one. Use whichever fits the job.

What it can do

• Drive ComfyUI from four agents (Claude Code, Codex, Gemini CLI, Qwen Code) off one shared core. GLM via Claude Code is covered too. (docs/AGENTS.md)
• ~90-tool MCP driver. The agent operates ComfyUI directly: generate, build / edit / validate graphs, queue, download models, manage VRAM, read logs, diagnose.
• Per-model "mega-brain": 75 prompt-recipe entries covering 68 named models, distilled from official sources (image, video, audio, 3D), split by family so each file reads whole in one call; the agent auto-pulls the right recipe when you name a model, so it prompts each one in its own dialect.
• Knows where each model runs: a full index of all 163 library models (recipe / utility / template-only), local vs API.
• Hardware-aware model selection: detects your VRAM, RAM, and free disk, then recommends the variant that fits (fp8 / offload / multi-GPU / quant) and refuses a download that won't fit, before wasting the bandwidth.
• 18 enhancement and utility tools: upscale / restore (Real-ESRGAN, SUPIR, SeedVR2), frame interpolation (FILM, RIFE), segmentation / depth / pose (SAM3, BiRefNet, Depth Anything), plus restoration chains.
• 581-template library (and 94 official Subgraph Blueprints, reusable subgraph bricks) as the source of truth, plus fetch any shared workflow by hash and a model shootout (run a prompt through many models small, pick the winner, then scale up).
• Assembles new workflows from parts: decomposes a task into stages, mixes templates and blueprint subgraphs, and wires the nodes correctly (output-to-input by type, with converters where needed), validated against /object_info before running. Not a preset runner.
• Expert color + custom nodes (new in v2): ships ComfyUI-OCIO
  • nine Nuke-style OpenColorIO nodes (Read / Write a still, sequence or video, grade in ACES, write ProRes / EXR, preview on an on-node player, fully color-managed) - and the agent knows each node's I/O plus the field guide to building a custom node pack. (docs/NODE_LIBRARY/ocio.md, docs/BUILDING_NODES.md)
• Starts ComfyUI for you: when the server is down, the agent launches it headless in the background and generates (no need to open the app first); to peek, you open http://127.0.0.1:8188 in a browser. For an unattended pipeline the start policy is configurable per project (env vars or a .comfyui-agent.json), so it never blocks on a prompt.
• GUI bridge + persistence: the agent writes graphs into your ComfyUI canvas, and SAVES every workflow it builds or runs to ComfyUI's workflows folder, so you can open it later from the Workflows sidebar (an API generation alone leaves no trace on the canvas).
• Stays current on its own: check_updates.py diffs the template repo and reads the blog RSS; an optional weekly task adds recipes for new models and pushes them. (docs/UPDATING.md)
• Three model skills that work outside ComfyUI too: seedance, minimax-h3, krea. Vendor prompting knowledge that also applies on the vendor's own app or API ships as its own skill rather than being buried in the ComfyUI reference. minimax-h3 owns the H3 three-field prompt format, <d> dialogue and the quant and acceleration ladder; krea owns the fork between Krea's hosted API and its open weights, FLUX.1 Krea Dev, and why the only ComfyUI pack for Krea Realtime is a 0-star lead rather than a route. The first of them, seedance: ByteDance's Seedance video models run on Dreamina, Jimeng and the BytePlus API as well as in ComfyUI, so their prompting knowledge ships as its own skill: the three task types and the one word that switches between them, the reference label syntax, the timestamp storyboard, the real-person character formula, and a failure table for face drift, duplicate characters and extension seams. Distilled from ByteDance's own guides. (shared/seedance/SKILL.md)
• Portable and idempotent: one installer, auto-detects your agents, re-runnable. Apache-2.0, no vendored third-party code (everything heavy is fetched at install).

The four-layer stack

Plus a GUI bridge: the agent writes graphs to <ComfyUI>/user/default/workflows/, you open them in the built-in Workflows sidebar and tweak them. No extra "agent panel" node required.

See docs/LAYERS.md for each layer, and docs/AGENTS.md for the per-agent matrix.

The template library is the source of truth

The kit clones the official Comfy-Org/workflow_templates and builds a compact lookup index so the agent can match any request to the right template. 581 templates (plus 94 official Subgraph Blueprints, reusable subgraph bricks) span every task, image, video, 3D, audio, utilities:

It knows every model's dialect

Each generative model rewards a different prompt approach: SDXL wants comma tags, FLUX wants natural-language sentences, video models want camera and motion direction, audio models want genre/tempo/instruments, and negative-prompt support varies wildly. The kit ships MODELS.md, a per-model prompting reference distilled from official sources (each maker's docs and model cards, docs.comfy.org, and the per-model templates from the anthropic-claude node). When you name a model in a request or a workflow, the agent reads that model's entry first and prompts it correctly.

Covered today (75 recipe entries, 68 named models): FLUX.1/.2 + Kontext, Z-Image, Boogu, Mage-Flow, Qwen-Image/Edit, SDXL, SD1.5/3.5, HiDream, Ideogram, Nano Banana Pro/2, Seedream, Recraft, GPT-Image, Grok, Reve, Kandinsky, BRIA, OmniGen, Chroma, Krea 1/2, ERNIE-Image, FireRed/LongCat/ChronoEdit/JoyAI Image Edit (edit), Capybara, Bernini-R, Anima, NewBie, PixelDiT, Ovis-Image, Lens, Quiver, Wan 2.1-2.7, LTX-2.3/2 Pro, Hunyuan Video, SVD, Kling, Veo, Sora, Seedance, Luma, Runway, MiniMax, PixVerse, Vidu, Pika, HeyGen (avatar video), HappyHorse, HuMo, SCAIL-2, Stable Audio, ACE-Step, ElevenLabs, ChatterBox, Sonilo, Hunyuan3D, Tripo, Rodin, Meshy. Plus a separate Enhancement and utility section (not prompt-driven, settings not prompts): upscalers and restorers (Real-ESRGAN, SUPIR, SeedVR2, FlashVSR, Topaz, Magnific), frame interpolation (FILM, RIFE), conditioning helpers (SAM3, BiRefNet, Depth Anything, DWPose, MoGe, IP-Adapter, LivePortrait, Mediapipe), and video object removal (VOID). Anything else falls back to the template library.

Full model index: every model in the library and exactly what the kit has for it (recipe / utility / template-only): docs/MODEL_INDEX.md.

Coverage table: every model and whether a prompt recipe is ready

✅ recipe = a dedicated, up-to-date prompting guide reached through the index in MODELS.md, which names the family file under shared/comfyui/MODELS/ that holds it. 🔧 tool = an enhancement/utility note (settings, not prompts). Table last reviewed: 2026-08-06.

One table, columns aligned to the widest row (the video models).

Niche models still without a recipe (very new, thin docs) run from their template and borrow the closest family's approach; see docs/MODEL_INDEX.md for the full per-variant breakdown.

Prerequisites

• One or more agent CLIs on PATH: Claude Code (claude), Codex (codex), Gemini CLI (gemini), Qwen Code (qwen)
• Node.js (node + npm)
• git, Python 3
• A local ComfyUI install (Desktop or source), comfy.org

Install

Claude Code: one-command plugin

Claude Code users can add the kit straight from the marketplace, no clone needed:

/plugin marketplace add SlavaSexton/ComfyUI-Agent-Kit
/plugin install comfyui@comfyui-agent-kit

That registers the local comfyui-mcp driver (launched with npx, no manual npm step) and loads the full comfyui skill (the 75-entry recipe brain + the docs). You still need a local ComfyUI on http://127.0.0.1:8188; the skill fills in your machine block on the first task. Plugins are Claude Code only, so for Codex / Gemini CLI / Qwen Code use the multi-agent installer below.

Every agent: the installer

Windows (PowerShell):

git clone https://github.com/SlavaSexton/ComfyUI-Agent-Kit.git
cd ComfyUI-Agent-Kit
./install.ps1 -ComfyUIPath "E:\path\to\ComfyUI"   # installs for every agent CLI found on PATH

Linux / macOS:

git clone https://github.com/SlavaSexton/ComfyUI-Agent-Kit.git
cd ComfyUI-Agent-Kit
./install.sh --comfyui-path /path/to/ComfyUI       # installs for every agent CLI found on PATH

The installer runs the shared machine setup once (MCP package, templates, in-graph nodes), then auto-detects which of claude / codex / gemini / qwen are installed and wires each one. It is idempotent, re-run it any time. Limit the targets with -Agents claude,gemini / --agents claude,gemini. Flags: -SkipTemplates / --skip-templates (skip the ~900MB template clone), -SkipNodes / --skip-nodes. Per-agent details and the GLM note are in docs/AGENTS.md.

First run on a new machine

After install, start ComfyUI, then in an agent session tell it to run the bootstrap once (docs/BOOTSTRAP.md): it detects your GPUs, VRAM, RAM, free disk, paths, and installed models via the MCP health_check, fills the machine-specific block in the skill, and does a smoke-test generation. After that, just ask for media. On Claude/Codex the skill auto-activates on ComfyUI keywords; on Gemini/Qwen the knowledge is loaded as the extension's context.

Optional: in-graph LLM key

Only needed if you want a workflow to enrich prompts without the agent in the loop (e.g. an unattended pipeline):

setx CLAUDE_API_KEY "sk-ant-..."   # then restart ComfyUI

See docs/NODES.md. When you are driving, the agent writes prompts directly, better and free.

Layout

ComfyUI-Agent-Kit/
├── install.ps1 / install.sh         top-level: shared setup + auto-detect agents + run adapters
├── shared/
│   ├── comfyui/                     SKILL.md + MODELS.md + comfy_client.py  (one source of truth)
│   └── tools/gen_quick_index.py     rebuild the template lookup index
├── agents/
│   ├── claude/   install.ps1/.sh    -> ~/.claude/skills/comfyui + claude mcp add + CLAUDE.md
│   ├── codex/    install.ps1/.sh    -> ~/.agents/skills/comfyui + ~/.codex/config.toml
│   ├── gemini/   install.ps1/.sh    -> ~/.gemini/extensions/comfyui (gemini-extension.json + GEMINI.md)
│   └── qwen/     install.ps1/.sh    -> ~/.qwen/extensions/comfyui (qwen-extension.json + QWEN.md)
├── docs/AGENTS.md                   per-agent matrix (how each connects) + GLM note
├── docs/MODEL_INDEX.md              every model in the library and what the kit has for it
├── docs/EXAMPLE_WORKFLOWS.md        notable shared workflows (model shootouts, restoration) + fetch helper
├── docs/UPDATING.md                 stay current: check_updates.py (templates diff + blog RSS) + the loop
├── docs/BOOTSTRAP.md / LAYERS.md / NODES.md
├── ATTRIBUTION.md                   credits for fetched third-party pieces
├── CHANGELOG.md                     curated history of notable changes (Keep a Changelog)
├── CONTRIBUTING.md                  what the kit accepts, the Teaching standard, house rules
├── LICENSE                          Apache-2.0 (this kit's original files)
└── NOTICE                           attribution + trademark notice (Apache-2.0 section 4d)

What is and isn't in this repo

In the repo (original work, Apache-2.0): the skill, the client, the installer, the index generator, the docs, the generated visuals. Fetched at install time from their own sources (not redistributed here): the comfyui-mcp package, the node-building skills, the workflow templates, and the in-graph Claude nodes.

Credits and thanks

Our own companion pack: ComfyUI-OCIO - nine Nuke-style OpenColorIO nodes - is by Slava Sexton, this kit's author (Apache-2.0). It is credited to him wherever the kit uses, recommends, or builds on it; see ATTRIBUTION.md.

This kit stands on excellent open-source work. It is a thin wiring layer over these projects, and the heavy lifting is theirs. Huge thanks to:

• ComfyUI by comfyanonymous / Comfy-Org, the engine everything runs on.
• comfyui-mcp by artokun, the MCP driver (Layer 2) that lets the agent operate ComfyUI with structured tools.
• comfyui-custom-node-skills by jtydhr88 / Terry Jia, the node-building skills (Layer 4).
• workflow_templates by Comfy-Org, the template library that is the source of truth.
• comfy-skills by Comfy-Org, whose output-node validation guard and multi-reference compositing technique this kit adapts (for the local stack) in SKILL.md and ADVANCED.md.
• comfyui-anthropic-claude by alexmunteanu and comfyui_claude_prompt_generator by PauldeLavallaz, the in-graph Claude nodes (Layer 3).

v1.1.0 builds on more excellent work. Thanks also to:

• Prompt Relay by Gordon Chen, Ziqi Huang, and Ziwei Liu (S-Lab, NTU), the training-free temporal prompt-routing method (arXiv 2604.10030).
• ComfyUI-PromptRelay and ComfyUI-SUPIR by kijai, the ComfyUI ports this kit recommends and drives.
• LTX Director 2.0 by WhatDreamsCost, the LTX-2.3 timeline-editor node.
• Z-Image-Turbo Fun-ControlNet-Union by alibaba-pai (PAI), plus the LTX-2.3 model and HDR IC-LoRA by Lightricks.
• Real-ESRGAN by Xintao Wang and the BasicSR team, and SUPIR by the XPixel Group (Fanghua Yu et al.), the upscale and restore models. Note: the SUPIR weights are non-commercial.

Field techniques in wide community use lean on:

• KJNodes by kijai (LTX-2.3 NAG, GGUF loading, chunked feed-forward, multi-guide), ComfyUI-CacheDiT by Jasonzzt (inference caching), ComfyUI-MelBandRoFormer (audio stem separation), ComfyUI-Frame-Interpolation by Fannovel16 (FILM), comfyui-inpaint-cropandstitch (Flux.2 masked inpaint), and GAP LTX 2.3 Motion by GeekatplayStudio (lipsync / storyboard / long audio).
• ComfyUI-Flux2Klein-Enhancer by capitan01R, the training-free multi-reference identity-transfer node suite for FLUX.2 Klein. Note: PolyForm Noncommercial license (commercial use needs a separate license).
• Smart Image Crop and Stitch by HallettVisual, an auto-sized crop/stitch node pair for high-res inpainting and detail edits (Apache-2.0).

v2.5.0 leans on more people's work. Thanks also to:

• Anima ControlNet-LLLite by kohya-ss, the control and inpainting patches that give the Anima base model depth, lineart, pose, scribble and masked editing (repacked for ComfyUI by Comfy-Org). Note: non-commercial license, inherited from the Anima base weights.
• krea2_style_reference and ComfyUI-Krea2-Ostris-Edit by ostris, the Krea 2 Turbo image-style-reference LoRA and the instruction-edit nodes this kit documents, plus the krea2-detail-enhancer-edit-lora by reverentelusarca.
• JoyAI-Image by JD (jd-opensource), the instruction image-edit model now supported natively in ComfyUI core (Apache-2.0).

Full per-component licensing is in ATTRIBUTION.md. If anything here misattributes your work, open an issue and it will be fixed.

Contributing

Read CONTRIBUTING.md first. It says what the kit accepts, what it does not and why, and the bar any knowledge entry has to clear. An issue before a large PR is welcome and usually faster.

License

Apache-2.0, see LICENSE and NOTICE. Releases up to v3.1.0 were MIT and that grant stands. Third-party components keep their own licenses.