ConstantineB6/comfy-pilot
comfy-pilot is an open-source MCP (Model Context Protocol) server designed specifically for ComfyUI, the popular node-based image generation tool. It acts as a bridge between AI agents and ComfyUI, allowing large language models and other AI systems to directly view, edit, and execute node-based workflows. The server exposes a comprehensive set of MCP tools that cover visual graph manipulation, node creation, workflow execution, and even terminal access to the ComfyUI environment. This enables AI-driven automation of complex image generation pipelines, making it possible for agents to design, debug, and run workflows without manual intervention. Key features include the ability to edit the node graph programmatically (edit_graph), place nodes with automatic collision avoidance, center the view on specific nodes, and retrieve detailed status information about the ComfyUI instance. The embedded terminal allows agents to execute shell commands within the ComfyUI environment, facilitating advanced operations like installing custom nodes or managing the filesystem. The project also includes Claude Code skills specifically tailored for ComfyUI node development, making it easier for developers to create and test new nodes with AI assistance. comfy-pilot is implemented as a ComfyUI plugin that starts an MCP server alongside the standard web interface. It uses WebSocket connections to synchronize state between the server, the ComfyUI backend, and connected clients. The server is designed to be lightweight and non-intrusive, with memory monitoring and a focus on avoiding disruptions to the user's manual workflow. The project includes unit tests and CI workflows, ensuring reliability and facilitating contributions. The tool is particularly useful for scenarios where an AI agent needs to autonomously create or modify image generation pipelines. For example, a coding assistant could generate a ComfyUI workflow from a natural language prompt, iteratively refine it based on user f
- Category
- Developer Tools
- Pricing
- Open Source
Problem solved
AI agents and developers who wish to automate and control ComfyUI workflows programmatically face a gap: existing APIs are limited to basic queueing and execution, while the node graph is typically manipulated through a visual UI. This makes it hard for AI systems to create, edit, or explore complex workflows without manual steps. comfy-pilot solves this by providing a full-featured MCP server that exposes all graph editing, execution, and system control capabilities as structured tools that any MCP-compatible AI agent can call. This enables seamless integration of AI into the ComfyUI ecosystem, saving time and reducing errors in tasks like pipeline generation, batch processing, and workflow optimization.
Use cases
- An AI agent automatically generates a Stable Diffusion workflow from a natural language description, places nodes logically, and executes it to produce images.
- Developers use Claude Code or other AI assistants to debug and modify existing ComfyUI workflows by editing node connections and parameters via MCP tools.
- Batch processing: an AI agent iterates over hundreds of prompts, dynamically adjusting the workflow for each one to achieve varied artistic styles.
- CI/CD pipelines for image generation: comfy-pilot is used in automated testing to validate custom ComfyUI nodes or workflows by running them and checking outputs.
- AI-assisted node development: using the included Claude skills, a developer builds a new custom node by having AI write the Python code and then test it live in a running ComfyUI instance.
- Remote workflow manipulation: a headless ComfyUI server is controlled entirely via an AI agent using comfy-pilot, with no need for the browser UI.
Audience
- Developers
- ML Engineers
- Data Scientists
- Researchers
Pros
- Full graph editing and execution control via MCP tools, enabling AI agents to autonomously design and run workflows.
- Includes visual helpers like automatic collision avoidance and view centering, making programmatic node placement reliable.
- Lightweight plugin design that runs alongside ComfyUI without disrupting the user interface.
- Supports unit testing and CI workflows, demonstrating a commitment to code quality and maintainability.
- Comes with specialized Claude Code skills for developing custom ComfyUI nodes, accelerating the extension ecosystem.
Cons
- Requires running an additional MCP server process, which adds some operational overhead.
- Currently tightly coupled to ComfyUI; cannot be used with other image generation platforms.
- Dependent on the MCP protocol being adopted by the target AI agent, limiting compatibility to MCP-aware assistants.
Official website: https://github.com/ConstantineB6/comfy-pilot
Canonical LambdaBase page: https://www.lambdabase.com/tools/constantineb6-comfy-pilot