Skip to Content
DocumentationPipelineBuilding Pipelines

Building Pipelines

You’ve successfully built your first pipeline in the 2-Minute Win. You’ve done two transformations and felt the power.

Now let’s build on that confidence and create your own pipeline from scratch.

You Already Know the Basics

From your 2-minute win, you already know:

  • ✅ How to load data into an Input node
  • ✅ How to add utilities to the canvas
  • ✅ How to configure utility settings
  • ✅ How to run a pipeline and see results

Everything else is just variations on these themes.

Creating a New Pipeline

1. Open the Pipeline Builder

  • Dashboard: Go to /dashboard/pipeline and click New Pipeline
  • Demo: Go to /demo/pipeline (no sign-in required)

You’ll see the Zero State canvas:

[Input Node] ──────> [Output Node]

This is your starting point. Not a blank canvas — a ready-to-use structure.

2. Load Your Data

Click the Input node and paste your JSON.

Or:

  • Click Upload to load a file
  • Click Fetch from URL to load from an API
  • Drag and drop a JSON file

3. Add Your First Utility

Find it in the left sidebar:

  • Browse by category (Convert, Merge, Structure, Cleanup, etc.)
  • Use the search box to filter utilities
  • Click utility name to see description

Add it to the canvas:

  • Drag it onto the canvas
  • It auto-connects to the nearest compatible node
  • Position it where you want

💡 Pro tip: Utilities auto-connect when you drag them near existing nodes. No manual wiring needed for simple pipelines!

4. Configure the Utility

Click the utility node to open its configuration panel on the right side.

Each utility has different settings:

  • Text fields — for search terms, field names, expressions
  • Dropdowns — for selecting modes, strategies, or formats
  • Toggles — for enabling/disabling options
  • JSON editors — for complex configuration like field mappings

💡 Pro tip: Click “What does this do?” in the configuration panel to see a description and examples.

5. Run and Verify

Click the Run control in the toolbar.

Watch the results:

  • Running state appears in the UI
  • Each step can show its output after execution
  • Output node shows the final result

Click any step to see its output at that point. This is how you debug!

Building Complex Pipelines

Chaining Multiple Utilities

Add utilities one after another. Each utility receives the output of the previous one.

Example: Clean messy API data

Input → Clean JSON → Filter Fields → Rename Keys → Output

Each step transforms the data further. You can click any step to see what it produces.

Branching Pipelines

One output can feed multiple inputs:

          → Filter Fields → Output
Input ──┼
          → Remove Fields → Output

This lets you try different transformations on the same data.

Merging Data

Some utilities (Deep Merge, Diff) need two inputs:

Input 1 ────┐
             ├──> Deep Merge ───> Output
Input 2 ────┘

Add a second Input node and connect both to the utility.

Select multiple nodes (Shift+Click or drag selection) then click Group to create a Group node.

This is purely visual — it helps organize your canvas but doesn’t affect execution.

Common Operations

Undo / Redo

Made a mistake?

ShortcutAction
Ctrl+ZUndo last pipeline change
Ctrl+YRedo

The pipeline builder tracks up to 50 changes.

Copy / Paste

ShortcutAction
Ctrl+CCopy selected node(s)
Ctrl+VPaste node(s)
Ctrl+DDuplicate selected node(s)

Deleting Nodes

Select a node and press Delete or Backspace.

Reconnecting

Drag a connection to a different handle to rewire it. The old connection disappears.

Canvas Navigation

  • Pan: Click and drag on empty canvas
  • Zoom: Mouse wheel or pinch gesture
  • Fit to screen: Click the fit button in the toolbar

Saving Your Work

Demo Mode

Pipelines auto-save to your browser’s localStorage. They persist between sessions but only on your device.

Dashboard Mode

Pipelines save to the cloud with full version history:

  • Auto-save: Changes save automatically
  • Manual save: Click Save (Ctrl+S) to create a named version
  • Version history: View and restore previous versions

Troubleshooting

Pipeline not working?

→ Check connections: Make sure nodes are connected → Check configuration: Click each step and verify settings → Check for errors: Failed steps show an error state — click to see the error → Try running again: Sometimes a second run clears transient issues

Common Issues

“Step produces no output”

  • Check if the step’s configuration is valid
  • Verify the input data matches what the utility expects
  • Look for error messages in the step node

“Connections won’t connect”

  • Make sure you’re dragging from an output handle (right side) to an input handle (left side)
  • Some utilities only accept certain connection types
  • Try connecting to a different handle

“Pipeline is slow”

  • Large datasets take longer to process
  • Complex pipelines with many steps take more time
  • Check the step indicators — they show processing progress

Next Steps

Now that you can build pipelines:

🎉 You’re ready! You know everything you need to build pipelines. Start transforming your data!

Quick Reference

Utility categories:

  • Convert — CSV ↔ JSON, format conversions
  • Merge — Combine data sources
  • Structure — Pick/remove fields, flatten/nest
  • Cleanup — Clean JSON, normalize, truncate
  • Analysis — Aggregate, summarize, diff
  • Generation — Mock data, compute
  • Parsing — Parse logs

Keyboard shortcuts:

  • Ctrl+Z — Undo
  • Ctrl+Y — Redo
  • Ctrl+S — Save (dashboard only)
  • Delete — Remove selected node
  • Ctrl+C/V — Copy/Paste
  • Ctrl+D — Duplicate
Last updated on
ConceptsRunning Pipelines