Runtime Executions

App page

Runtime Executions is the run history. Every deployed data pipeline run writes a record here: scheduled, started by hand, or started by a webhook or a hosted form.

Where to find it

Open Monitor > Runtime Executions.

• The Last execution badge in a project's header opens that project's latest run here.
• A link to one run opens this page with that run expanded.
• A link copied from the Pipelines view opens the same data pipeline and run.
• Project names in the list open the project on the data pipeline that run used.

What you see

The page has two views, chosen with the Pipelines and All runs buttons at the top. Both use the same filters, and the figures at the top follow them: Projects, Runs in range, Success rate, Rows in, Rows out and Avg duration.

Pipelines

Pipelines opens first. The left list has one row per data pipeline that ran in the range. A project with more than one data pipeline shows each as Project / Data pipeline. Each row shows the status of its last run, how long ago it started, and the share of finished runs that completed.

Select a data pipeline to see on the right:

• Its schedule and next run, and how its last run started and in which environment.
• Open pipeline, which opens it on the canvas.
• Success, Avg duration, Runs, and Failed or completed with errors for the range. A run still in progress is not counted in Success.
• A bar per run for the last 30 runs, its height the run's duration. Green completed, amber completed with errors, red with stripes failed, blue running. Select a bar to open that run.
• Run history: the runs, newest first, ten at a time. Select a row to expand the run. Show N more runs shows the next ten. While not every run in the range is loaded, the table header says how many are, for example Showing 50 of 120 runs, beside Load more runs.

The figures count runs a webhook or a hosted form started from the newest deliveries only. When a range holds more, a note under the figures says how many deliveries were counted.

All runs

All runs is one list of every run for your account, sortable by each column. Expand a run to see the run log, per-node metrics, and per-node details and payload samples where the node records them.

An expanded run shows the same detail and the same actions in both views: refresh status, download log, download payloads, download debug payloads, Ask Milo and delete.

Reading a run

• Status is one of the following.

  Status	What it means
  Queued	Waiting for a free slot on the runtime.
  Running	In progress.
  Completed	Every node finished.
  Completed with errors	The run finished and part of it did not happen. Either a target refused one or more rows and no error edge from that target took them, or a node failed and its Execution Policy said continue, so the run skipped that node and everything downstream of it. Open the run log to see which node.
  Failed	The run stopped.
  Stopped	Cancelled, or stopped by a run duration limit.
  Skipped	A scheduled job run that did not start. The row shows the reason, for example plan limit reached when the monthly scheduled-run limit is used up.
  Cancelling	Stop was requested. The run has not stopped yet.
  Rescheduled	The run stopped at a connection's rate limit and its job runs it again from the last checkpoint. The run detail says when. Not a failure.

• A run a webhook or a hosted form started names it under the project, for example Hosted form: Contact us. The run detail also shows its environment.

• Duration is the time from start to the final status.

• Rows In and Rows Out in the list are the rows the run's sources read and its targets wrote. A row a target refused is not counted as written. They read "None" when the run recorded no row counts, which is not the same as zero.

• Per-node row counts show how many rows entered and left each node.

• Payload sample shows rows that passed through the node, when the node's sample-payload setting is on.

• Logs are timestamped lines from the runtime, indented by node.

Which cards are open

Expanding a run opens Node Metrics and leaves the rest collapsed: What this run cost, Pipeline Steps, the Run Log, Log Events and the per-node result cards. Each header carries a count or a headline figure, so a collapsed card still says whether it is worth opening.

Open and closed cards are remembered per run while the page is open, so opening the Run Log on one run does not open it on the others.

Run Log and Log Events

A run has two log surfaces, and they hold different things.

Run Log is the runtime's account of the run: one line per node as it starts, finishes and routes its rows. Every run has it.

Log Events is filled only by the Log and Profile nodes. It holds structured records with a severity, so it can be filtered by level and searched. A flow with neither node shows no events.

What this run cost

The first card in an expanded run, when the run has these figures. It opens collapsed, with two headline figures on its header: peak memory and whether anything was written to disk.

• Memory budget and Peak memory used: the run's limit and its highest reading. The badge reads Comfortable, Slowed down, or Hit its ceiling.
• Written to disk: data a node set aside to stay within the budget. "Stayed in memory" means no node needed to.
• Source chunks: how many chunks the run read and the average rows in each. "Not chunked" means no source was read in chunks.
• Rows per statement: how many rows each database write carried, for the target that carried the fewest. "One per row" means a target made one round trip per row, which is slow.
• Waiting to start: time source chunks spent queued behind other chunks.
• Rate limit wait: time requests waited for a connection's pace, with how many waited and the longest single wait. Shown only when a request waited.
• Slowest measured node: the node with the lowest rows per second.

The node table adds two columns for these runs: Throughput in rows per second, and Disk. Both read "Not measured" or "None" when there is no figure, which is not the same as zero. When a node's requests waited for a connection's pace, the table also shows Rate limit wait for each node.

Run report

A run that was eligible for an execution report shows a Run report row with what happened to it:

Badge	What it means
Sent	The report was accepted by the mail server.
Not requested	No report was asked for. The account has no report settings, reports are off, or this data pipeline is not subscribed.
Cannot send	A report was due and could not be sent. Email DataLug support at support@datalug.io with the run.
Refused by the mail server	Every recipient was rejected.
Failed	The send failed, or the run did not carry enough information to decide.

A run with no Run report row is one whose status is not final, or that has no report record.

Turn reports on per data pipeline on the canvas, in the right panel's Properties tab under Execution reports, and set the account's recipients and detail level under Manage Account > Notifications. See Run notifications.

Scrolling the tables

Every table in a run's detail (node metrics, pipeline steps, and the input and output samples) sits in its own scroll box with a fixed height. The column headers stay in place while the rows move under them, and a wide table scrolls sideways. Select a table once to give it focus, then use the arrow keys, Page Up and Page Down to move through it without a mouse.

Each table shows the first 100 rows. When there are more, a line under the table says how many were left out.

The downloads are not capped. The CSV and JSON buttons in a node's header download every row the run captured. A sample holds up to 100 rows, so most samples show in full.

Queued runs

A runtime runs a limited number of data pipelines at once. Past that limit, a run waits in a short queue and shows as Queued until a slot frees. When the queue is also full, the run is refused straight away and the caller is told when to retry.

Nodes that hold rows

Some nodes hold rows until the whole source has arrived: Fork in Buffer All mode, and Join or Union placed downstream of more than one source. Large inputs to these nodes make a run slower. Use Streaming mode on Fork, or filter rows upstream, when a flow does not need every row at once. The run log names any node that held a large input.

A run fails with a message naming the node when a held row contains a value that cannot be stored, such as a Map, a Set or a function produced by a Script node.

Retention

Log entries and run records are kept for the platform retention period.

Common tasks

Find a run

Filter the list by date range and by status.

• Time range: Last 1h, Last 24h, Last 7d, Last 30d or Today. The selected range is highlighted. Editing either date by hand clears it.
• Status: opens a list of every status in the table above. Tick one or more. The button shows the selection, and Show every status clears it.

Runs load for the selected range, the last 30 days by default, 50 at a time, including runs a webhook or a hosted form started. Select Load more runs for the next ones. The figures in the Pipelines view count every run in the range, loaded or not.

The project search and the status filter also narrow the Pipelines list to the data pipelines with matching runs.

Share a run

Expand the run and copy the page address. The link opens this page with that run expanded, in the view it was copied from. The browser's Back button steps back through the data pipelines and runs you opened.

Ask Milo about a failed run

On a failed run, select Ask Milo in the failure summary. The Milo Assistant opens with the run attached and a question ready to send. See Milo in the app.

Go from a run back to the node

A node's name is a link, both in Node Metrics and in the per-node result card. It opens the project on the data pipeline canvas, selects that node and shows its inspector.

When the node type has a configuration screen, the link opens it. When it has none (Start, Variable, Transform, SQL, Data Profile), the link selects the node and shows its inspector. The hover text says which of the two it does.

The name is not a link when the run has no project recorded. When the node has been deleted from the canvas, the link opens the canvas with nothing selected.

Stop a run on a Private Agent

A Private Agent runs on your own network and accepts no inbound connections. A stop request reaches the agent the next time it checks in, every few seconds, so the run shows Cancelling until it stops.

Node row counts, payload samples and logs for a Private Agent run stay on the agent until the run finishes. The run detail shows the status and says where the detail is held.

Troubleshooting

A run you expect is not in the list:

• The run was a Quick Debug Run on the canvas. Those do not write Runtime Execution records.
• The schedule has not fired yet. Check the Jobs page for the next run time.
• The runtime is offline. Open Environments, select the environment, and check the agent's heartbeat on the Overview tab.

Related

• Jobs: the schedules that start runs.
• Run performance: reading a run's performance figures.