Comfy Pilot
Talk to your ComfyUI workflows. Comfy Pilot gives Claude Code direct access to see, edit, and run your workflows — with an embedded terminal right inside ComfyUI.
Why?
Building ComfyUI workflows means manually searching for nodes, dragging connections, and tweaking values one at a time. With Comfy Pilot, you just describe what you want:
- "Build me an SDXL workflow with ControlNet" — Claude creates all the nodes, connects them, and sets the parameters
- "Look at the output and increase the detail" — Claude sees your generated image and adjusts the workflow
- "Download the FLUX schnell model and set up a workflow for it" — Claude downloads the model and builds a workflow from scratch
No copy-pasting node names. No hunting through menus. Just say what you want.
Installation
CLI (Recommended):
comfy node install comfy-pilot
ComfyUI Manager:
- Open ComfyUI
- Click Manager → Install Custom Nodes
- Search for "Comfy Pilot"
- Click Install
- Restart ComfyUI
Git Clone:
cd ~/Documents/ComfyUI/custom_nodes && git clone https://github.com/ConstantineB6/comfy-pilot.git
Claude Code CLI will be installed automatically if not found.
Requirements
- ComfyUI
- Python 3.8+
Features
- MCP Server - Gives Claude Code direct access to view, edit, and run your ComfyUI workflows
- Embedded Terminal - Full xterm.js terminal running Claude Code right inside ComfyUI
- Image Viewing - Claude can see outputs from Preview Image and Save Image nodes
- Graph Editing - Create, delete, move, and connect nodes programmatically
Demo
https://github.com/user-attachments/assets/325b1194-2334-48a1-94c3-86effd1fef02
Usage
- Restart ComfyUI after installation
- The floating Claude Code terminal appears in the top-right corner
- The MCP server is automatically configured for Claude Code
- Ask Claude to help with your workflow:
- "What nodes are in my current workflow?"
- "Add a KSampler node connected to my checkpoint loader"
- "Look at the preview image and tell me what you see"
- "Run the workflow up to node 5"
MCP Tools
The MCP server provides these tools to Claude Code:
| Tool | Description |
|---|---|
get_workflow | Get the current workflow from the browser |
summarize_workflow | Human-readable workflow summary |
get_node_types | Search available node types with filtering |
get_node_info | Get detailed info about a specific node type |
get_status | Queue status, system stats, and execution history |
run | Run workflow (optionally up to a specific node) or interrupt |
edit_graph | Batch create, delete, move, connect, and configure nodes |
view_image | View images from Preview Image / Save Image nodes |
search_custom_nodes | Search ComfyUI Manager registry for custom nodes |
install_custom_node | Install a custom node from the registry |
uninstall_custom_node | Uninstall a custom node |
update_custom_node | Update a custom node to latest version |
download_model | Download models from Hugging Face, CivitAI, or direct URLs |
Example: Creating Nodes
Create a KSampler and connect it to my checkpoint loader
Claude will use edit_graph to:
- Create the KSampler node
- Connect the MODEL output from CheckpointLoader to KSampler's model input
- Position it appropriately in the graph
Example: Viewing Images
Look at the preview image and describe what you see
Claude will use view_image to fetch and analyze the image output.
Example: Downloading Models
Download the FLUX.1 schnell model for me
Claude will use download_model to download from Hugging Face to your ComfyUI models folder. Supports:
- Hugging Face (including gated models with token auth)
- CivitAI
- Direct download URLs
Terminal Controls
- Drag title bar to move
- Drag bottom-right corner to resize
- − Minimize
- × Close
- ↻ Reconnect session
Architecture
┌─────────────────────────────────────────────────────┐
│ Browser (ComfyUI) │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
│ │ xterm.js │ │ Workflow State │ │
│ │ Terminal │ │ (synced to backend) │ │
│ └────────┬────────┘ └────────────┬─────────────┘ │
│ │ WebSocket │ REST API │
└───────────┼────────────────────────┼────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────┐
│ ComfyUI Server │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
│ │ PTY Process │ │ Plugin Endpoints │ │
│ │ (claude CLI) │ │ /claude-code/* │ │
│ └─────────────────┘ └──────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│ │
│ ▼
│ ┌──────────────────────────┐
└──────────▶│ MCP Server │
│ (stdio transport) │
└──────────────────────────┘
Files
__init__.py- Plugin backend: WebSocket terminal, REST endpointsjs/claude-code.js- Frontend: xterm.js terminal, workflow syncmcp_server.py- MCP server for Claude Code integrationCLAUDE.md- Instructions for Claude when working with ComfyUI
Troubleshooting
"Command 'claude' not found"
Install Claude Code CLI:
macOS / Linux / WSL:
curl -fsSL https://claude.ai/install.sh | bash
Windows (PowerShell):
irm https://claude.ai/install.ps1 | iex
Windows (CMD):
curl -fsSL https://claude.ai/install.cmd -o install.cmd && install.cmd && del install.cmd
MCP server not connecting
The plugin auto-configures MCP on startup. Check ComfyUI console for errors, or manually add to ~/.claude.json:
{
"mcpServers": {
"comfyui": {
"command": "python3",
"args": ["/path/to/comfy-pilot/mcp_server.py"]
}
}
}
Terminal disconnected
Click the ↻ button to reconnect, or check ComfyUI console for errors.
License
MIT