Projects

App page

The Projects page is where you create, import, open, and manage projects. Each project is a workspace containing data pipelines and saved actions.

Where to find it

Select Projects in the main menu. Opening a project shows its data pipelines on the canvas. Links from Jobs and Runtime Executions open the project on the data pipeline they name.

What you see

Open projects and data pipelines

The projects you have open sit in a row at the top of the page, each showing how many of its data pipelines are open. The data pipelines inside the project you are on sit in a second row, below the project card and directly above the canvas.

• A data pipeline opens once. Selecting one that is already open brings you to its existing tab rather than opening a second copy of it. Opening a data pipeline from the Project Explorer follows the same rule.
• Selecting a project returns you to the data pipeline you were last on in it.
• A data pipeline shows an environment badge when all of its schedules target the same one. Data pipelines with no schedule, or with schedules in more than one environment, show no badge.
• Close a tab with the cross on it, or by middle-clicking it. With the tab focused, Delete and Backspace also close it.
• Closing a project closes every data pipeline open under it. It asks first when that is more than one. Unsaved edits are kept on your computer either way.
• The project row's + opens a project. In the pipeline row, + creates a new data pipeline in the project you are on, and the arrow beside it opens one that is not open yet.
• The count at the right, "3 projects, 7 pipelines open", lists everything open grouped by project, and offers Close other pipelines and Close other projects.
• Eight data pipelines can be open at once. Opening a ninth closes the one you opened first. A data pipeline with unsaved changes is never the one closed.
• Your open tabs are saved to your account, so they return when you sign in again, including on another computer.

Where you are on the canvas

A breadcrumb above the canvas reads Projects > project > data pipeline, followed by the number of steps in it. The data pipeline you are on is shown in a tinted pill. Select its name to rename it. When the row is short, the project name is shortened first.

Above the canvas, beside the Save button, a badge reports whether your work is safe. It reads Saved with the time, or Unsaved changes.

The canvas toolbar

The toolbar above the canvas holds the Saved badge, Autosave, Save, Undo, Redo and data pipeline history, then a pod with the view, zoom, display and auto arrange controls.

When the canvas is too narrow for every control, the least used ones move into the More menu (...) at the end of the pod, in this order:

1. Auto arrange
2. Display toggles: error paths, edge record counts and branch colours
3. Center on nodes, Zoom to fit and Reset zoom
4. Data pipeline history
5. Autosave
6. Zoom in and Zoom out

Save, Undo and Redo always stay on the toolbar. Last of all the Saved badge is hidden; the dot on the Save button still shows unsaved changes. The canvas is narrower when the side panels are wider, the window is smaller, or Milo is open.

In the More menu, the arrow keys, Home and End move between items, Enter chooses one, and Esc closes the menu. A toggle shows a tick while it is on.

Project detail header

• Shows the project name, clickable to edit, with when it was last updated. The description is shown on the Projects page; edit it through the pencil beside the name.
• "Last execution" badge links to the associated Runtime Execution record.
• The Project Explorer in the sidebar lists the project's data pipelines, connections and saved actions.

The canvas right panel

The panel on the right of the canvas has three tabs.

Tab	What it holds
Node Config	The selected node's settings, and the preview of what it produced.
Runs	This data pipeline's recent runs, the live log, and per-node metrics. Pick a node above Node Metrics to show only its row.
Properties	Settings and facts about the data pipeline itself.

The Properties tab has these rows:

Row	What it does
General	Name, identifier, when it was last saved, and the latest version number.
Error paths	Whether this data pipeline shows error paths when it opens.
On failure	What a node does when it fails with no error wire of its own.
Execution reports	Turn run report emails on for this data pipeline, and pick the report type.
Schedule	The jobs that run this data pipeline, with a link to the Jobs page.
Variables	How many variables the project has, and where to edit them.

Error paths, On failure and Execution reports are editable here. The other three summarise settings you change on their own pages.

Only the jobs that run this data pipeline appear; the project's other schedules do not.

Ports and edges

Under its header, a node card has an In lane down its left edge and an Out lane down its right edge, each headed In or Out. Every port has its own row in its lane, and its dot sits on the edge of the card: inputs on the left edge, outputs on the right edge. Input dots are grey and output dots are indigo, on every node. The error output keeps its own red dot. A filled dot has an edge attached; a hollow dot is free. Hover a port, or move focus to it, to see its name beside it. Router routes show their values and Fork outputs show their labels.

Nodes with several inputs or outputs, such as Join, Router, Fork, Filter and Approval, show one row per input or output. A card grows taller only when its lanes need more rows. A Union always shows one dashed empty input after the ones in use. Connect to it and another appears. On a Fork, the + beside Out adds an output.

An edge leaves its output to the right, turns down or up in a lane of its own close to the node it leaves, and enters its input from the left. Corners are rounded. Edges from one node each turn in their own lane, so they do not run over each other. Where two edges cross, the lower one shows a small gap. Edges are grey until a run colours them.

The band along the bottom of each card has two halves.

The left half shows whether the node is set up, using the same checks as the Node Config panel and the run check:

• Configured (green, with a check mark): nothing left to fix.
• Not set up (amber): a setting the node needs is not made yet, such as its connection or saved action.
• 1 issue, 2 issues (amber, with a warning sign): a setting is filled in but not valid, or the linked connection is missing.

Hover it to see the issues the Node Config panel lists. Select it to open Configure.

The right half shows the node in the run selected for the canvas:

• Not run yet: no run is shown.
• Running, with a spinner and the rows so far, such as 820 rows.
• The rows in and out, such as In 1,725 and Out 1,610. A source shows only Out, and Start shows Done. A side with no rows is left out.
• 12 failed (amber), then the rows in and out: the node finished but sent rows to its error output.
• The failure reason (red), or Failed, with a Fix link.

Hover it to see when the node ran and its full totals: in, out, failed, and the rows on each output. Select it to open the Runs tab on that run, with Node Metrics showing only this node. Select Show all nodes beside the node's name to see every node again. Fix opens Configure when the node also has a setup issue, and the Runs tab otherwise.

Hover an edge or a port to bring its edges forward; the others fade until you move away. A hovered or selected edge also shows its label, such as a route value or a Fork output label.

Branch colours, a button in the canvas control pod beside # (or in the More menu on a narrow canvas), changes only how ports are drawn. On a node with more than one output, each output and its edges get their own colour instead of indigo and grey. It is off until you turn it on, and the choice stays in this browser.

A node's ports always show in the same order.

To add a step after a node, hover the node and select + in its header, beside copy and delete.

Connect, disconnect and reconnect

• Connect: press on an output and drag. The inputs that can take the connection pulse, and the rest fade. Near an input, the line snaps to it. Release to connect.
• Drop on a node: release anywhere over a node and the line goes to that node's free input nearest the pointer. If the node has no free input that fits, nothing connects and a short message says so.
• Cancel: press Esc, or release over empty canvas.
• Disconnect: hover an edge and select the x at its midpoint. You can also select an edge and press Delete or Backspace.
• Reconnect: press on the input end of an edge and drag it to another input. Release over empty canvas to remove the edge.
• Keyboard: ports are buttons. Select an output, then an input, to connect them.

An input that already has an edge does not take a second one, except on a Union. Undo brings back an edge you removed.

Error paths on the canvas

Every node except Start has an error output: the last row of its Out lane, under a dashed line, with a red dot. Wire it to say where that node's rows go when it fails. The rows that arrive carry the original row plus the error message, the id of the node that failed and its name.

• The error dot is a red ring with a red dot in the middle, wired or not.
• A wired error output shows a chip under the card naming its handler, such as -> Notify ops, instead of drawing the wire. Select the node or switch error paths on to see the wire itself.
• Several nodes can feed one handler. The handler shows how many.
• Error wires are dotted red. When a run fails on that node, its wire turns bolder.
• Catching a failure does not make the run succeed. The run still ends as failed.

The eye button in the canvas control pod, beside the zoom controls, shows or hides error paths. On a narrow canvas it is Error paths in the More menu. That choice is yours and stays in this browser. A data pipeline can carry its own default under Properties > Error paths, which applies when the data pipeline opens and which your own toggle overrides.

With error paths shown, the handlers sit in a band below the flow marked Error lane. Auto-arrange places a node that only receives failures in this lane, clear of the main flow.

Edges after a run

When you run a data pipeline, its edges show how the run went.

• A green edge led to a node that completed.
• A red edge led to a node that failed, or left a node that failed. It also carries a ! marker at its midpoint.
• An edge the run did not reach stays grey. While the node it leads to is still running, it glows softly.
• A pill on the edge shows the rows that travelled along it, such as 0 rows, 1 row or 1,544 rows. Each output of a Fork or Router shows its own count, and an error edge shows the failed rows sent to it.
• Edges inside a Loop Container show the rows across all of its iterations.
• An Approval's approved, rejected and timeout edges show their counts once the run continues.
• Runs on a Private Agent show colours and counts once the run finishes.
• Older runs show colours without counts.
• Each node shows its own totals in the right half of the band along the bottom of its card, such as In 1,725 and Out 1,610, and 3 failed in amber when its error output sent rows.

Colours and counts follow the run selected for the canvas and update as nodes finish.

The # button in the canvas control pod shows or hides the counts. Colours and the failure marker stay either way. The choice is yours and stays in this browser.

On failure

A node with nothing wired to its error output follows the data pipeline's On failure setting, under Properties > On failure:

Setting	What happens
Stop the run	The run stops at that node. This is the default.
Skip the node and continue	The run carries on. Nodes downstream of the failed one are still skipped, because they have no rows.
Send to a handler	The failed rows go to one node you pick, exactly as a wired error output would.

A node with its own error wire, or its own failure setting under Advanced > Execution Policy, ignores this setting.

Choosing Send to a handler and leaving the node blank stops the run, and the panel says so. A handler you later delete also stops the run rather than losing the rows.

Catching a failure this way does not make the run succeed. The run still ends as failed.

Common tasks

Create a new project

1. 1
   Select New Project

   On the Projects page.
2. 2
   Choose a creation method

   Empty (the default) creates a blank project with a Start node. Starter creates a project from a template with sample nodes and edges. Describe opens Milo on the canvas with your description ready to draft a data pipeline, when Milo is available on your account. Import creates a project from a project file downloaded from DataLug.
3. 3
   Fill in the form

   Project name, First data pipeline (optional, defaults to the project name) and an optional description.
4. 4
   Create the project

   The canvas opens and you can start adding nodes.

Add a data pipeline

Use the + in the pipeline row above the canvas, or the + beside Data Pipelines in the Project Explorer. The new data pipeline is listed as Untitled until you name it with the pencil beside it.

Export or import a project

• Export: in the Projects table, use the download action to save a project file.
• Import: in New Project, choose Import and upload the file.

Import creates a new project. Connection references may be left out or remapped, so check each node's connection and environment settings before you run it.

Clone a project or data pipeline

• Project cloning creates a new project with its own nodes and actions and no references to the original.
• Data Pipeline cloning copies the nodes, edges and actions within a project and adds "Copy" to the names. Connections are not copied. Actions are copied as new records, so two data pipelines never share one.

Edit a project's name and description

On the project detail page header, click the project name itself or the small pencil icon next to it. The Edit Project modal opens with Name and Description fields. The pencil and the name are disabled for roles without write access (Viewer and Auditor).

Related

• Flow rules: what the canvas and a save refuse.
• Jobs: deploy and schedule a data pipeline.