Building Pipelines

This guide walks you through creating a pipeline from scratch.

Prefer a head start? Use AI Draft in the editor to generate a first-pass pipeline from plain-language instructions, then open it as a new pipeline and refine it here.

Add an input

Every pipeline starts with an Input node. This is created automatically when you open the pipeline builder. Click the input node and paste or type your source JSON.

You can add multiple inputs for utilities that require two inputs (e.g., Deep Merge, Diff).

Add a step

Click Add Step to browse available utilities. Select one to add it to the canvas. Each step represents one transformation operation.

Popular starting utilities:

Clean JSON — remove nulls, empty strings, trim whitespace
Find & Replace — search and replace across keys and values
Pick Fields — keep only the fields you need
Flatten / Nest — convert between flat and nested structures

Connect nodes

Drag from the output handle (right side) of one node to the input handle (left side) of another. The connection defines the data flow.

A step can receive input from:

An Input node
Another step’s output
Multiple sources (for utilities like Merge or Diff that take two inputs)

Configure the step

Click a step node to open its configuration panel on the right side. Each utility has different settings:

Text fields — for search terms, field names, expressions
Dropdowns — for selecting modes, strategies, or formats
Toggles — for enabling/disabling options
Forge JSON Editors — for complex configuration like field mappings

Run the pipeline

Click the Run control to execute the entire pipeline. When idle it shows a play icon; while running it switches to a spinning sync icon. Results appear in step and output views after execution.

Tips

Step ordering

Steps execute in dependency order (topological sort), not visual position. A step runs only after all its input connections have been processed.

Error handling

If a step fails, it shows an error state. The error message appears when you click the step. Downstream steps that depend on the failed step will not execute.

Undo / Redo

The pipeline builder has its own undo/redo system (up to 50 entries). Changes to node positions, connections, and configurations are all tracked.

Shortcut	Action
Ctrl+Z	Undo last pipeline change
Ctrl+Y	Redo

Re-running

After changing configuration, click Run again. The engine re-executes the entire pipeline from scratch with the updated settings.

Each run starts fresh from the input data — pipeline execution is stateless. Previous run results don’t affect the next run.

Using utilities from the Store

You can add any utility from the Store to your pipeline:

Browse utilities at /store
Click Use in Pipeline on a utility detail page
You’ll be taken to the pipeline builder with that utility pre-added

Secondary inputs

Some utilities require two inputs:

Utility	Primary input	Secondary input
Deep Merge	Base document	Document to merge in
Diff	Left document	Right document to compare

Connect each input to a different source (Input node or upstream step).