Docs · AI server setup

What to install on your ComfyUI box

heapedit's AI features run against a ComfyUI server you operate. This page is for whoever provisions that server: which custom node packs and model files each feature needs, where the files go, and how to verify the install.

Base setup

1. Install ComfyUI — a recent build. The Image Edit feature needs the Flux.2 node set that ships in ComfyUI core; update if your build predates it.
2. Start it so a browser page may call it:

   python main.py --enable-cors-header --listen

   --listen binds beyond localhost so other machines on your network can reach it. If heapedit itself is served over HTTPS, browsers block calls to an http:// server — put ComfyUI behind an https reverse proxy or tunnel in that case.
3. In heapedit: AI → AI Settings → paste the server address → Test Connection → Refresh from server (fills every model dropdown with what your server actually has).
4. Install the ComfyUI-Manager extension on the server — the node packs below are easiest to install by searching Manager for the node class names listed here.

Verify any node pack without leaving the terminal — GET <server>/object_info/<NodeClass> returns the node's schema if installed, {} if not:

curl -s http://127.0.0.1:8188/object_info/ReActorFaceSwap | head -c 100

The one model almost everything shares

Most generative features run on SD 1.5 and default to Realistic Vision V6.0 B1 (from Civitai — download both variants):

Any SD 1.5 checkpoint pair works — the checkpoint is selectable in AI Settings after Refresh from server. The inpainting variant is what Generative Fill / Expand / AI Remove / Sky Replace use.

Per-feature requirements

Composite features reuse the rows above: Replace Sky = AI Select + Generative Fill, AI Remove = AI Point Select + Generative Fill, Product Photo Cleanup = Remove Background (the centering/shadow half is client-side).

Suggested install order

1. SD 1.5 checkpoints + RMBG → unlocks Generative Fill/Expand/Generate Layer, Remove Background, Product Cleanup.
2. The segment-anything2 pack → unlocks AI Select and Replace Sky.
3. ReActor + an upscale model → Face Restore, Face Swap, Upscale.
4. Everything else per taste.

Custom workflows — bring your own ComfyUI graph

Every feature above ships with a working built-in graph, but you're not stuck with it. Each feature can hold several named presets in AI Settings → Custom workflows ("fast draft", "high quality", …) — whichever preset is selected in the dropdown is what runs, and Built-in workflow is always there to switch back to.

1. In AI Settings, open the feature's Custom workflows section and click + New preset.
2. In ComfyUI, build (or open) the graph you want, then export it — plain Save or dev-mode Save (API Format) both work, heapedit auto-detects and converts.
3. Paste the JSON into the preset's text box.
4. Tell heapedit which node is which, either way:
   • Automatic — right-click each relevant node in ComfyUI → Title → rename it HEAPEDIT_<SLOT> using the slot names shown for that feature. Re-export and paste; it resolves with no further steps.
   • Manual — don't want to rename nodes? Once pasted, heapedit lists every node in your graph (id, type, title) with a dropdown per required slot — just assign them directly. Mappings are stored per preset.
5. A status line shows ✓ Valid once every slot resolves, or names exactly what's still missing.

Copy built-in as template on any feature copies a ready-to-use starting graph (already using the right titles) to your clipboard — the fastest way to see the expected shape before editing or building from scratch.

Example: a custom Colorize workflow

Colorize is the simplest feature to swap — it only needs three slots: HEAPEDIT_IMAGE (grayscale input in), HEAPEDIT_PROMPT (the color description), and HEAPEDIT_OUTPUT (the result out). Say you want to swap the built-in ControlNet recolor graph for, say, an SDXL-based one:

1. Build your graph in ComfyUI: a LoadImage node feeding your pipeline, a CLIPTextEncode node taking the prompt, and a SaveImage node at the end.
2. Right-click the LoadImage node → Title → rename it to HEAPEDIT_IMAGE.
3. Right-click the CLIPTextEncode node that should receive the color prompt → rename it to HEAPEDIT_PROMPT.
4. Right-click the final SaveImage node → rename it to HEAPEDIT_OUTPUT.
5. Export as API-format JSON — the relevant nodes now carry the titles heapedit looks for:

{
  "1": { "class_type": "LoadImage", "_meta": { "title": "HEAPEDIT_IMAGE" }, "inputs": { "image": "" } },
  "4": { "class_type": "CLIPTextEncode", "_meta": { "title": "HEAPEDIT_PROMPT" }, "inputs": { "text": "", "clip": ["3", 1] } },
  "11": { "class_type": "SaveImage", "_meta": { "title": "HEAPEDIT_OUTPUT" }, "inputs": { "images": ["10", 0] } }
  // ...the rest of your graph's nodes, wired however you like
}

Paste that whole JSON into the Colorize preset's text box; heapedit finds the three titled nodes automatically and shows ✓ Valid. Run Colorize from the tool and it uses your graph instead of the built-in one — same inputs and outputs, whatever pipeline you want in between. Other features just have more slots (Face Swap adds a SOURCE_IMAGE slot for the face to swap in, Generative Expand adds a PAD slot, AI Point Select adds a POINT_X slot, etc.) but the mechanism is identical — tag the nodes, paste the graph, done.

Gotchas

• First run per model is slow. A cold model load can take minutes (and looks like a hang). heapedit's poll timeout is configurable in AI Settings (Poll timeout); warm each feature once after installing.
• "Succeeded" but nothing changed usually means a node silently skipped its work (e.g. ReActor logs No faces found → skipping). The server's /internal/logs/raw and /history/<prompt_id> endpoints show per-node logs and the executed graph.
• A stale custom workflow shadows the built-in graph. Workflow presets live in the browser's localStorage — if a feature runs a graph you don't expect, check AI Settings for an active preset on that feature and switch it back to Built-in.
• Model dropdowns look wrong? They're guesses until you hit Refresh from server in AI Settings — do that once per new server.

For deployers: turning features off

If your server won't support some features (or you don't want to offer them), block them at build time instead of letting them fail at runtime — see src/core/aiFeaturePolicy.ts in the repo. Features are tiered basic (analyze/clean up existing pixels) and advanced (generate new content):

# hide the whole advanced tier
VITE_AI_BLOCKED_TIERS=advanced npm run build

# or block specific features
VITE_AI_BLOCKED_FEATURES=faceSwap,relight npm run build

Blocked features grey out in the AI menu; every run path (built-in graphs, custom workflows, and the AI tools in the rail) is gated centrally, with the tool-rail tools reporting the block as their normal error toast.