Run notifications
A run notification is an email sent when a data pipeline run finishes. It names the data pipeline and the environment, says whether the run succeeded, and reports how long it took and how many records moved.
Notifications are off until you turn them on, in two places:
- The account holds the recipients, how often to send, and how much detail each email carries.
- Each data pipeline has its own on/off switch. A data pipeline you have not switched on sends nothing, even when the account is configured.
Set up the account
Go to Manage Account > Notifications and find the Run Notifications card.
| Setting | What it does |
|---|---|
| Enable run notification emails | The account-wide switch. Off means nothing sends, whatever a data pipeline says. |
| Default recipients | Who receives the report. |
| How often | Every run or Failed runs only (the default). |
| Environment policy | Per environment, whether a data pipeline may turn failure reports off. |
| Detail level | Summary report, Detail report, or Summary, detail on failure. |
| Nodes to leave out of the report | Node types to omit. Every type is listed; six start ticked. |
Daily digest, Attach rejected records (CSV), Run starter and Subject line are shown on the card but cannot be changed. Every run sends its own report.
Nodes to leave out of the report
Every node type is listed. A ticked type is omitted from the report.
Six start ticked: Log, Cache, Message, Variable, Data Profile and Notification. They can appear many times in a flow and say little about what a run did. Untick any of them to include it, and tick any other type to hide it.
A run with no node left to report still sends. The report then carries the run's own status and totals.
Recipients
The account address always receives reports and cannot be removed. It starts as the account owner's address, and it is checked when you save, even with the switch off.
Add as many more as you need:
- An email address. Any address, including one outside your organization.
- The account owner. Follows the owner, so it keeps working after a handover.
- A group. Every member of one of your organization groups, resolved when the email is sent, so adding someone to the group adds them to the reports.
Each recipient chooses which outcomes reach them, so an on-call group can take failures while a reporting mailbox takes everything.
Environment policy
Each environment is either:
- Allowed (the default): each data pipeline in it decides for itself.
- Required for failures: a failed run always reports, even if that data pipeline has notifications switched off. Use it for production.
A run that completed with errors counts as a failure here.
Only failures are forced. A successful run in a "Required for failures" environment follows the data pipeline's own setting.
Changing the recipients or the detail level needs the Owner or Admin role. Turning a single data pipeline's notifications on or off needs the Builder role.
Turn it on for a data pipeline
Open the project and the data pipeline on the canvas, then use either:
- The right panel's Properties tab, under Execution reports. This is also where you choose the report type for this data pipeline.
- The Actions menu item Email a report when this workflow runs. A check mark beside it means notifications are on.
The switch belongs to the data pipeline you have open, not to the project. A project with three data pipelines can report on one of them.
Report type for a single data pipeline
The Properties tab's Report type list overrides the account's detail level for this data pipeline. Leave it on Account default to follow the account.
The list is locked until reports are on for the data pipeline.
What the emails contain
Every report names the data pipeline, the project, the environment, the DataLug address
that sent it, the run id, and when the run finished. The subject line carries the
DataLug address without https://. Quote the run id if you contact support about a run.
Summary report: how long the run took, how many records were written, how many nodes completed, and the run path. For a failure it also names the node the run stopped at and the error.
Detail report: adds a timeline of when each node ran and how long it took, and a card per node with its record counts, its start and end times, and, for the node that failed, the error. For a failed run it also lists the nodes that never ran.
Summary, detail on failure: a summary report for a successful run and a detail report for a failed one.
Times in the emails are UTC, and the emails say so.
Reports count records, not bytes. A failure does not say which record it stopped on; the node's error message carries what the connected system returned.
A node card does not name the connection or the saved action the node used. Called by gives the step number of the node that passed the records in, not its name.
When notifications do not send
A run produces no email when any of these is true:
- The account switch is off, or the account has no recipient address.
- The data pipeline's own switch is off.
- The run succeeded and How often is set to Failed runs only.
- The run was stopped or cancelled by hand. A cancelled run is not a failure.
A run that completed with errors does send under Failed runs only, and the report names the node.
A report is sent once, after the run reaches its final state.
Email wording
There are six report emails: a summary and a detail version for each outcome (succeeded, completed with errors, failed). Their wording and layout are the same for every account.
Runtime unreachable alerts
A run report tells you how a run finished. A runtime alert tells you that a runtime has stopped answering, so scheduled runs on it will not start.
Each runtime sends DataLug a regular heartbeat. When the heartbeats stop for longer than your threshold, DataLug emails the address you set, or the account's Owners when the recipient is blank. The email reports:
- How long the runtime has been unreachable.
- The threshold that triggered the alert.
- When the last heartbeat arrived, in UTC.
- An hour-by-hour strip over the last day, showing when the heartbeats stopped.
- Likely causes and three things to check.
When several runtimes in your account stop answering at once, you get one email listing all of them.
Set the recipient and the threshold
Open Manage Account > Notifications and find the Operational alerts card. Turn on Send operational alert e-mails, then set Recipient email, Threshold (minutes) and Cooldown (minutes). Leave Recipient email blank to email every Owner of the account. The cooldown is how long DataLug waits before repeating an alert for a runtime that is still unreachable. Select Save alerts.
While the card is on, its recipient is also added to webhook dead-letter e-mail and connection health e-mail. Owners and Admins receive those whether or not the card is on.
Turning the switch off stops these alerts for your account. Run reports have their own switch and are not affected.
Runtime alerts are always sent, even when run reports are held back.