comfy-pilot is an open-source MCP (Model Context Protocol) server designed specifically for ComfyUI, the popular node-based image generation tool. It acts as a bridge between AI agents and ComfyUI, allowing large language models and other AI systems to directly view, edit, and execute node-based workflows. The server exposes a comprehensive set of MCP tools that cover visual graph manipulation, node creation, workflow execution, and even terminal access to the ComfyUI environment. This enables AI-driven automation of complex image generation pipelines, making it possible for agents to design, debug, and run workflows without manual intervention. Key features include the ability to edit the node graph programmatically (edit_graph), place nodes with automatic collision avoidance, center the view on specific nodes, and retrieve detailed status information about the ComfyUI instance. The embedded terminal allows agents to execute shell commands within the ComfyUI environment, facilitating advanced operations like installing custom nodes or managing the filesystem. The project also includes Claude Code skills specifically tailored for ComfyUI node development, making it easier for developers to create and test new nodes with AI assistance. comfy-pilot is implemented as a ComfyUI plugin that starts an MCP server alongside the standard web interface. It uses WebSocket connections to synchronize state between the server, the ComfyUI backend, and connected clients. The server is designed to be lightweight and non-intrusive, with memory monitoring and a focus on avoiding disruptions to the user's manual workflow. The project includes unit tests and CI workflows, ensuring reliability and facilitating contributions. The tool is particularly useful for scenarios where an AI agent needs to autonomously create or modify image generation pipelines. For example, a coding assistant could generate a ComfyUI workflow from a natural language prompt, iteratively refine it based on user feedback, and execute it to produce images. Developers can extend the server's capabilities by adding new MCP tools or leveraging the built-in skills framework for Claude Code.
Solves
AI agents and developers who wish to automate and control ComfyUI workflows programmatically face a gap: existing APIs are limited to basic queueing and execution, while the node graph is typically manipulated through a visual UI. This makes it hard for AI systems to create, edit, or explore complex workflows without manual steps. comfy-pilot solves this by providing a full-featured MCP server that exposes all graph editing, execution, and system control capabilities as structured tools that any MCP-compatible AI agent can call. This enables seamless integration of AI into the ComfyUI ecosystem, saving time and reducing errors in tasks like pipeline generation, batch processing, and workflow optimization.