A ComfyUI custom-node package that connects Pi Agent reasoning with practical workflow analysis, model discovery, project planning, screenplay tools, reference manifests, tutorial compilation, document export, and Kdenlive-first editorial handoff.
Creator: Alan D. Guice (Badgids)
License: GPL-3.0
Release: 0.1.18
This project is designed to be powerful without being confusing. The documentation uses plain language, short steps, and concrete examples. Technical details are kept intact instead of being hidden or oversimplified.
ComfyUI can generate images, video, speech, music, sound, and many other kinds of media. Large projects become difficult when they need many workflows, references, prompts, scripts, shot lists, and output files.
ComfyUI Pi Agent helps organize that work. It can:
- inspect and explain ComfyUI workflows;
- validate and safely repair basic workflow structure;
- discover installed model files without inventing filenames;
- understand safetensors and GGUF as component formats;
- build model-aware prompt packages;
- create reference-asset manifests for characters, voices, audio, mood boards, and storyboards;
- create and parse Fountain screenplays;
- create screenplay breakdowns and starter shot lists;
- create complete project directory structures and documentation;
- compile one or more workflows into a thorough ComfyUI-native tutorial;
- export Markdown, DOCX, Fountain, and JSON files;
- prepare a Kdenlive-first editorial package with a portable timeline fallback;
- call Pi through its JSONL RPC mode for structured/headless features when Pi is installed and configured;
- run the real interactive Pi CLI in either the ComfyUI left sidebar or its bottom panel through a native controlling PTY, with Pi reasoning, tools, slash commands, menus, and errors rendered directly;
- keep a structured Chat view as a secondary/fallback interface, with reasoning and tool activity visible by default and independently hideable;
- recognize, explain, inspect, plan, create, and safely edit workflows for ComfyUI-MiniMaxH3-Director when that pack is installed;
- recognize, explain, inspect, plan, create, and safely edit workflows for WhatDreamsCost-ComfyUI, including LTX Director, Prompt Relay, keyframes, IC-LoRA, audio, and its utility nodes;
- understand and operate ComfyUI-scene-camera-action, including SceneState staging presets, human/car acting, camera directing, and captured previz reference output;
- understand and operate ComfyUI-MiniMax-H3-Turbo, including its Turbo LoRA, dual-schedule 4-step sampler, strength tuning, low-VRAM mode, T2V/I2V use, and H3 joint audio/video constraints;
- dynamically load node-pack knowledge only when the current request or workflow needs it, instead of filling Pi's context window at startup;
- deterministically select a small set of task procedures so even weak local models are told how to stay on task, inspect before guessing, act, validate, and report evidence;
- preemptively write a durable continuity checkpoint around 82.5% context usage and immediately request Pi's native same-session compaction at the completed-turn boundary, preserving the active Pi session instead of using
/new; - generate deterministic semantic flowcharts and current Nodes 2.0 schematic node illustrations for documentation;
- capture real ComfyUI workflow/node screenshots for tutorials through an optional Playwright browser backend, with node captures retaining at least 300 CSS pixels of surrounding context on every side;
Every major tool is available as a normal ComfyUI node. The optional sidebar is disabled by default and is not required.
Version 0.1.18 adds same-session preemptive compaction, persistent Terminal session recovery, strict live-schema workflow generation, semantic documentation diagrams, real Playwright workflow/node screenshots, and the refreshed Pi Agent session UI on top of the existing production, tutorial, document, workflow-intelligence, and NLE toolset.
It does not bundle large AI model weights, third-party custom-node packs, Pi itself, Node.js, FFmpeg, or Kdenlive. It detects those tools when they are installed. Missing optional tools do not stop ComfyUI from starting.
The plugin never downloads anything during import.
Documentation starts here: Documentation home and navigation map. Every guide contains links to go up, back, forward, and to closely related guides.
Navigation guarantee: the automated test suite verifies that every Markdown documentation/procedure file is reachable from this README through local links and that those local documentation links resolve.
- Pi runtime and model-provider setup
- Small local model reliability
- Pi Agent sidebar
- Real Pi terminal
- Local LLM servers and Pi slash commands
- Workflow intelligence
- Model discovery, safetensors, and GGUF
- Dynamic integration context
- Preemptive same-session context compaction
- Image generation and editing profiles
- Music, speech, and audio profiles
- Project directory layout and asset organization
- References, mood boards, storyboards, and bibles
- Stories, books, Fountain, and screenplays
- Complete and incremental production compiler
- ComfyUI-native tutorial compiler
- Kdenlive and NLE handoff
- Architecture
- Compatibility
- Known limitations
- Security and path policy
- Development and testing
- Roadmap
- Example workflows
- Bundled Pi skills and procedures
- Project inventory
- Release notes
- Changelog
- Contributing
- Agent and development rules
- Security policy
- Code of conduct
- Third-party notices
- GPL-3.0 license
- Model-family profiles
Open a terminal in the custom_nodes directory inside your ComfyUI installation.
cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/Badgids/ComfyUI-Pi-Agent.gitcd C:\path\to\ComfyUI\custom_nodes
git clone https://github.com/Badgids/ComfyUI-Pi-Agent.gitRestart ComfyUI.
The core plugin has no mandatory third-party Python dependency. Two optional feature sets can be installed with the same Python interpreter/environment that runs ComfyUI:
# Existing hand-authored ASCII → SVG conversion with Ascidia
python -m pip install -e '.[diagrams]'
# Real ComfyUI workflow/node screenshots with Playwright
python -m pip install -e '.[screenshots]'
# Install both optional feature sets
python -m pip install -e '.[diagrams,screenshots]'If the screenshot backend cannot find an installed Chromium/Chrome browser, install Playwright's Chromium build once in that same environment:
python -m playwright install chromium- Download the project ZIP.
- Extract it into
ComfyUI/custom_nodes. - Make sure the directory contains this
README.mdand the top-level__init__.py. - Restart ComfyUI.
In ComfyUI:
- Double-click an empty part of the canvas.
- Search for
Pi Agent Status. - Add the node.
- Queue it.
The node reports the plugin version and whether the Pi executable was found.
Read the full installation guide when ComfyUI uses a portable Python build, Docker, WSL, or a nonstandard user directory.
Add and run:
Pi Agent Status
Pi is optional for the non-agent project tools. A missing Pi executable is reported as a normal status message, not a startup error.
Add and run:
Pi Model Inventory
The node asks ComfyUI for model filenames registered in categories such as checkpoints, diffusion models, text encoders, VAEs, LoRAs, ControlNets, and audio encoders.
- Save or copy a ComfyUI workflow JSON.
- Paste the JSON or enter its path in
Pi Analyze Workflow. - Run the node.
The result includes:
- workflow format;
- node count;
- link count;
- detected node types;
- detected input files;
- detected model-like filenames;
- missing live node classes;
- structural warnings and errors.
Add Pi Compile Project and enter a simple request, for example:
Create a three-minute animated short about a young pony who learns to cross a flooded creek.
The node creates a complete, organized project directory with writing, bibles, references, production, generated-media, editorial, quality-control, NLE, and delivery folders.
It labels the project planned, not complete. Media is only complete after the required workflows actually run and pass validation.
Add Pi Compile Tutorial.
Supply one workflow object, a list of workflow objects, or a list of workflow paths as JSON:
[
"/path/to/01_character_sheet.json",
"/path/to/02_storyboard.json",
"/path/to/03_video.json"
]The compiler creates:
- preserved original workflows;
- annotated tutorial workflows;
- stage-by-stage Markdown and DOCX guides;
- model and custom-node manifests;
- a Tutorial Controller workflow;
- a Project Overview workflow;
- validation and troubleshooting files.
The tutorial is completed entirely inside ComfyUI. No separate private WebUI is created.
- Pi Agent Status — finds Pi safely and reports plugin/runtime status.
- Pi Agent Prompt — sends a prompt to Pi through strict JSONL RPC mode; an optional workflow input enables lazy workflow-aware/node-pack-aware reasoning without dumping the full graph into context.
- Pi Analyze Workflow
- Pi Validate Workflow
- Pi Repair Workflow
- Pi Model Inventory
- Pi Model Resolver
- Pi Model Profiles
- Pi Prompt Package
- Pi Image Workflow Plan
- Pi Qwen Image Edit Plan
- Pi Krea 2 Edit Plan
- Pi Audio Workflow Plan
- Pi MiniMax H3 Director Status — detects the separately installed Director pack and its public nodes.
- Pi MiniMax H3 Director Plan — chooses FL2VA/ref2VA and validates duration/reference limits.
- Pi MiniMax H3 Director Workflow — creates a workflow from the installed Director pack's own current example workflow instead of fabricating a third-party graph.
- Pi Inspect MiniMax H3 Director Workflow — checks Director-specific model, VAE, prompt, reference, and timeline requirements.
- Pi Scene Camera Action Status — detects
SceneNode,ActingNode, andDirectingNode, installed presets, and the upstream staging skill. - Pi Scene Camera Action Plan — plans staging, human/car acting, duration, and optional directing.
- Pi Scene Camera Action Workflow — creates a safe base Scene → Acting → Directing previz chain without inventing recorded frontend state.
- Pi Inspect Scene Camera Action Workflow — checks the chain, SceneState structure, actor settings, and frontend-managed editing boundaries.
- Pi MiniMax H3 Turbo Status — detects the Turbo LoRA/sampler pack and its installed example workflow.
- Pi MiniMax H3 Turbo Plan — plans T2V/I2V/FLF, steps, LoRA strength, and low-VRAM mode.
- Pi MiniMax H3 Turbo Workflow — creates from the installed upstream Turbo example and can attach first/last image inputs by named sockets.
- Pi Inspect MiniMax H3 Turbo Workflow — checks the Turbo LoRA, dual-schedule sampler, scheduler, H3 model path, and AV requirements.
- Pi Reference Asset
- Pi Character Sheet Plan
- Pi Mood Board Plan
- Pi Storyboard Plan
- Pi Production Bible
- Pi Fountain Screenplay
- Pi Parse Fountain
- Pi Screenplay Breakdown
- Pi Shot List
- Pi Production Plan
- Pi Production Build Plan
- Pi Compile Project
- Pi Complete Production
- Pi Compile Tutorial
- Pi Tutorial Load
- Pi Tutorial Preflight
- Pi Tutorial Stage Select
- Pi Tutorial Stage Validate
- Pi Tutorial Note
- Pi Save Markdown
- Pi Export DOCX
- Pi Save JSON
- Pi Kdenlive Package
- Pi Show Text
The complete input and output reference is in docs/node-reference.md.
The custom node discovers Pi in this order:
- the explicit executable path entered in the node;
- the
PI_AGENT_EXECUTABLEenvironment variable; - the system
PATH.
Pi is started with --mode rpc. The Python client sends one JSON object per line and reads Pi events until the agent is fully settled.
Pi still needs a model provider. In the optional sidebar, Provider then Model are directly beneath the chat box. The Provider dropdown covers Pi's current built-in provider catalog plus local/custom providers; hosted models come from Pi's live available-model catalog, while local hosts populate their own reported models on selection. For llama.cpp router mode, ComfyUI-Pi reads the router's live /models catalog so configured-but-unloaded presets remain selectable. Unloaded or sleeping router models are explicitly woken. ComfyUI-Pi follows llama.cpp's router lifecycle by polling /models until the selected preset is loaded, then performs a lightweight routed /tokenize probe for that exact model before launching Pi. The user's configured chat timeout is the complete readiness budget. No private model name or personal timeout is hardcoded. Common local endpoints are automatic and endpoint editing is advanced/optional. Provider credentials and models are managed by Pi, not written into ComfyUI workflows.
Read docs/pi-runtime.md and docs/local-llm-slash-commands.md.
The plugin does not assume that a model is just one file. A usable model may require:
- diffusion or transformer weights;
- one or more text encoders;
- a vision encoder;
- a multimodal projector;
- a VAE or audio codec;
- a required edit LoRA;
- matching loader nodes;
- a variant-specific sampling profile.
Pi Model Resolver searches the filenames ComfyUI already knows. It can prefer safetensors or GGUF, but it does not pretend that two similar names are compatible.
The bundled profile registry includes:
- Qwen Image;
- Qwen Image Edit;
- Krea 2;
- Krea 2 Edit;
- FLUX.2 Klein;
- Z-Image;
- ACE-Step 1.5 and XL;
- Qwen3-TTS;
- Stable Audio 3;
- MiniMax H3 Director (FL2VA/ref2VA interoperability profile);
- WhatDreamsCost LTX Director (distilled/GGUF workflow interoperability profile).
- Scene Camera Action 3D previz/SceneState interoperability profile;
- MiniMax H3 Turbo 4-step LoRA/sampler interoperability profile.
These profiles describe capability and component expectations. Live ComfyUI node schemas and installed templates remain the authority for actual workflow construction.
Read docs/model-formats-gguf.md.
ComfyUI-Pi includes first-class interoperability knowledge for the separately installed ComfyUI-MiniMaxH3-Director node pack. It recognizes the Director, Preview Override, Retake Stitch, and Enhance Prompt nodes; understands FL2VA versus ref2VA; knows the model/VAE roles, reference limits, prompt formats, frame-grid rules, joint audio/video decode, retake behavior, and safe editing boundaries.
The integration does not copy the upstream GPL-3.0 Python/JavaScript code into this GPL-3.0 repository. When asked to create a Director workflow, ComfyUI-Pi locates the user's installed Director pack and starts from that pack's own current example workflow. If the pack is missing, Pi reports the missing dependency instead of inventing a fake workflow.
The knowledge is not injected at Pi startup. The dynamic integration router loads the MiniMax H3 Director guide only when the user asks about it, when a matching node ID appears in the attached workflow, or when the user explicitly calls its integration node.
Read docs/minimax-h3-director.md and docs/dynamic-integration-context.md.
ComfyUI-Pi also includes first-class knowledge for WhatDreamsCost-ComfyUI. It recognizes LTXDirector, LTXDirectorGuide, LTXDirectorCropGuides, LTXKeyframer, MultiImageLoader, LTXSequencer, SpeechLengthCalculator, LoadAudioUI, and LoadVideoUI. It understands the LTX Director timeline, Prompt Relay, first/middle/last guide frames, custom audio, audio inpainting, IC-LoRA reference workflows, Retake Mode, timeline save/load, and the upstream distilled and GGUF example-workflow paths.
When asked to create a WhatDreamsCost workflow, ComfyUI-Pi prefers the installed pack's own current example workflow and then uses live ComfyUI schemas for safe changes. It does not guess the large timeline frontend's private widgets_values indexes or blindly rewrite timeline_data.
This integration is also lazy: unrelated Pi conversations receive none of the WhatDreamsCost guide.
Read docs/whatdreamscost-comfyui.md.
ComfyUI-Pi recognizes SceneNode, ActingNode, and DirectingNode from ComfyUI-scene-camera-action. It understands the upstream SceneState/blockout contract, actor-aware spawn/layout rules, human/car acting stage, camera-cut directing stage, and the captured video/stage outputs used as previz references.
The integration deliberately does not manufacture recorded motion_data or directing_data: those are interactive frontend states owned by the upstream widgets. Pi can create and edit SceneState JSON, build the public three-node chain, inspect/repair the graph, and explain how to use the output in downstream generation.
Read docs/scene-camera-action.md.
ComfyUI-Pi recognizes MiniMaxH3TurboLoRA and MiniMaxH3TurboSampler from ComfyUI-MiniMax-H3-Turbo. It understands the required MODEL insertion, the custom dual video/audio schedule, simple/4-step starting profile, LoRA strength tuning, low_vram sharpness/VRAM trade-off, pruned/full H3 base support, and H3's 24 fps / 17k+5 constraints.
A Turbo request does not automatically load the separate MiniMax H3 Director guide. The two integrations are loaded together only when the user asks to combine them or the workflow actually contains both node packs.
Read docs/minimax-h3-turbo.md.
ComfyUI-Pi does not assume that the connected local LLM is large or highly capable. The host code keeps a short operating contract, classifies the current job deterministically, selects only a few matching bundled procedures, supplies a concrete completion rule, and resets stale hidden procedure context when the work changes.
This means a small model does not have to remember the whole plugin, choose from every skill, or infer basic rules such as "inspect the workflow before changing it". Large workflows, manuals, and project files remain available by path and are read only when the task needs their exact contents.
The design cannot make a very weak model reason like a much stronger model, but it removes avoidable ambiguity and moves routing, context control, and safety rules into deterministic code.
Read docs/small-model-reliability.md.
ComfyUI-Pi keeps Pi's starting context deliberately lean. Its supervised RPC subprocess disables Pi's automatic project context files, discovered extensions, discovered skills, prompt templates, themes, project trust, and Pi-side session persistence for that run. The tiny integration registry stays host-side and is not injected into the model at startup.
For each request, the router checks the user's message and any attached/current workflow. Only a matching node pack is loaded. Normal matched requests receive a compact, task-targeted integration summary; the full bundled guide is reserved for explicit comprehensive-guide/tutorial/deep-dive requests. Unrelated requests receive zero MiniMax H3 Director, WhatDreamsCost, Scene Camera Action, or MiniMax H3 Turbo knowledge. In sidebar chat, large active workflows are represented by a compact digest plus an on-demand local workflow file instead of pasting the entire graph into every turn.
ComfyUI-Pi also prevents previously injected pack knowledge from lingering indefinitely in stateful chat. When the relevant integration/project/workflow scope changes, it resets Pi's in-memory RPC session and rehydrates only a bounded clean user-visible transcript before injecting the new scope.
This keeps ordinary chat, writing, project planning, and unrelated ComfyUI work from paying the context-window cost of every supported node pack. See docs/dynamic-integration-context.md and docs/pi-runtime.md.
The tutorial compiler is intentionally native to ComfyUI.
It does not include:
- a private WebUI;
- a standalone tutorial website;
- a second server;
- an external launcher.
It creates normal ComfyUI workflows containing tutorial notes and groups. The generated Tutorial Controller workflow loads the tutorial manifest, performs preflight checks, selects stages, and validates progress.
A complete tutorial package may contain:
README.md
QUICK_START.md
QUICK_START.docx
COMPLETE_TUTORIAL.md
COMPLETE_TUTORIAL.docx
TROUBLESHOOTING.md
TROUBLESHOOTING.docx
tutorial.json
Tutorial_Controller.json
Project_Overview.json
original workflows
annotated workflows
stage guides
model manifests
custom-node manifests
validation reports
Read docs/tutorials.md.
Pi Compile Project accepts a simple request and creates a complete project structure. It is safe to use independently or as the beginning of a larger pipeline.
The compiler creates only starter documents and manifests. It does not falsely claim that ungenerated media exists.
Every generated project begins with START_HERE.md and a complete directory guide. Major creative assets have their own plainly labeled top-level directory:
00_PROJECT_ADMIN
01_STORY
02_SCREENPLAY
03_PRODUCTION_BIBLES
04_MOOD_BOARDS
05_REFERENCE_SHEETS
06_STORYBOARDS
07_SCENE_AND_SHOT_PLANS
08_PROMPTS
09_COMFYUI_WORKFLOWS
10_GENERATED_MEDIA
11_EDITORIAL_MEDIA
12_NLE_PROJECT
13_TUTORIALS_AND_DOCUMENTATION
14_QUALITY_CONTROL
15_DELIVERY
16_ARCHIVE
Every generated directory and subdirectory contains a README.md explaining exactly what belongs there. The compiler also creates asset-catalog.json and directory-map.json, so users and automation can locate assets without guessing.
The current project compiler emits:
- project brief and assumptions;
- Markdown and DOCX documents;
- starter Fountain screenplay for narrative requests;
- empty structured bibles;
- production and asset manifests;
- Kdenlive assembly-document placeholders;
- clear planned/partial/complete status boundaries.
Read the project directory layout guide and the production compiler guide.
Kdenlive is the default NLE target.
Pi Kdenlive Package accepts a production manifest and creates:
- a basic Kdenlive/MLT project;
- an OTIO-style JSON fallback;
- a timeline JSON file;
- a media map;
- a project profile;
- a Markdown assembly guide;
- a DOCX assembly guide.
The native Kdenlive writer is intentionally conservative. Complex effects, title templates, nested sequences, transitions, and version-specific features should be finished inside Kdenlive. The portable manifest and assembly guide remain available when a native project needs repair.
Read docs/kdenlive-nle.md.
The optional Pi Agent interface can live in the left sidebar or ComfyUI bottom panel and has two views:
- Terminal — the default when native PTY support is available. This is the real interactive
piCLI rendered inside ComfyUI, so Pi itself owns reasoning display, tool calls/results, slash commands, interactive menus, keyboard behavior, and streaming output. - Chat — the structured ComfyUI chat retained as a secondary/fallback view. It recovers final answers with Pi's authoritative RPC text command when necessary and shows reasoning/tool activity by default, with settings to hide either.
Directly below the active view are Provider then Model selectors. Local hosts are discovered only when selected; model IDs come from the live host rather than hardcoded model lists. Common endpoints are automatic and endpoint editing remains advanced/optional.
Terminal mode still uses ComfyUI-Pi's sparse context router. Pi starts with unrelated context/skill/extension discovery disabled and loads one explicit bridge extension that adds only the task procedure or node-pack knowledge needed for the current user request. The user's terminal input is not replaced by a giant generated prompt.
The preemptive continuity guard also remains active. At the configured 80–95% threshold (82.5% default), the Terminal bridge writes and verifies a bounded durable checkpoint and hidden same-session anchor. When the threshold is observed at turn_end, it immediately requests Pi's native ctx.compact() in the current Pi session. agent_end is prepare-only fallback state and agent_settled is used only if a prepared checkpoint still needs a safe fallback request. No /new is sent for compaction. Manual /compact and Pi overflow recovery keep their native same-session lifecycle while ComfyUI-Pi supplies/verifies the durable continuity payload.
Enable the optional interface and choose its placement in ComfyUI settings:
Pi Agent: Enable interface after restart
Pi Agent: Interface placement = Left sidebar | Bottom panel
There is no separate user-facing WebUI or second browser application. The real terminal is a controlling PTY supervised by the ComfyUI backend and rendered in the selected native ComfyUI panel. Placement changes take effect after refreshing the ComfyUI browser page.
Read Pi Agent sidebar, Real Pi terminal, and Local LLM servers and Pi slash commands.
This package follows these rules:
- No personal path is hardcoded.
- No model or dependency is downloaded during import.
- No API key is stored in a workflow by the plugin.
- Original workflows are preserved before repair or tutorial annotation.
- File writes use atomic replacement.
- Relative project paths are preferred.
- Missing models and nodes are reported instead of invented.
- Pi is launched with an argument array, never a shell command string.
- Optional failures do not stop ComfyUI from loading.
Read docs/security.md and SECURITY.md.
The repository uses Python's built-in unittest, so tests need no extra test framework.
From the project directory:
python -m unittest discover -s tests -vThe tests run without ComfyUI by using compatibility fallbacks.
Read docs/development.md.
From the cloned repository:
git pullRestart ComfyUI after updating Python or JavaScript files.
Review CHANGELOG.md before updating a production environment.
Created by Alan D. Guice (Badgids).
This project builds on the public extension systems and documentation of ComfyUI and Pi Agent. ComfyUI-Pi includes interoperability knowledge for separately distributed node packs including GPL-3.0 ComfyUI-MiniMaxH3-Director by seesee75-commits, GPL-3.0 WhatDreamsCost-ComfyUI by WhatDreamsCost, MIT ComfyUI-scene-camera-action by arturitu (whose upstream staging skill is Apache-2.0), and Apache-2.0 ComfyUI-MiniMax-H3-Turbo by Larryvrh. Their source code and weights are not bundled into ComfyUI-Pi. Third-party models, custom nodes, media, fonts, and model weights keep their own licenses. The GPL-3.0 license covers the original ComfyUI-Pi code and documentation in this repository. Third-party projects and assets keep their own copyrights and license notices. See THIRD_PARTY_NOTICES.md.
GNU General Public License v3.0. See LICENSE.