Log node
What it does
The Log node records a structured event for each row, or for the rows that match a condition: a level, a message and a set of fields. The events appear in the run's Log Events card. Rows pass through unchanged, so the node observes the flow without changing it.
Log node events are kept in the run log. To send run, error and audit events to an external logging service, see Manage Account > Log Destinations.
When to use
- You want to see what a node received, without adding a Load you later remove.
- You want row values, such as a customer id or an order total, recorded with each event.
- A flow logs at
debugwhile you build it, and you want those events dropped in scheduled runs.
Configure
Select the node and press Configure to open Configure Log. The steps are Name & level, Message, Fields, Advanced and Summary & preview.
Save and Done stay unavailable while a required setting is missing. Cancel discards the changes made since the dialog opened or was last saved. A message confirms each save.
- Level:
debug,info,warn,errororfatal. - Message: a template.
{{row.field}},{{exec.varName}}and{{sys.runId}}insert values. Insert field adds an upstream field at the cursor. An unclosed{{, an unknown namespace or an unknownsyskey shows as an error. A field the upstream schema does not list shows as a warning. Neither blocks saving. - Fields: name and value pairs recorded with the event. Add a field from the upstream node adds a row with the field's name and
{{row.field}}. A value that is only{{row.field}}keeps its type, so numbers stay numbers. Any other value is recorded as text.
Advanced
- Emission trigger:
Every row(default).When expression is truthy: the expression is a template, filled in per row. A comparison with>,>=,<,<=,=,==,!=or<>is evaluated; ordering comparisons are numeric. Any other text counts as true unless it is empty,false,0orno.Only at or above run's minimum level.
- Extra redaction keys: a comma-separated list of field names to hide, added to the standard list (
password,token,secret,api key,bearer,authorization). Matching is exact and ignores case.
Min log level on the run launch sets the lowest level recorded for that run. The default is info, so debug events are dropped unless you lower it.
Ports
| Port | Direction | Kind | Notes |
|---|---|---|---|
in | in | data | Rows to log. |
out | out | data | Every input row, unchanged. |
error | out | control | Receives the input rows when the node fails. |
Reading the events
Open the run and find the Log Events card. Each event shows its level, the node's name and the message. Expand an event to see its fields, with redacted values hidden. Filter by level, or search the list.
Logging failed rows
Connect a node's error port to a Log node to record why rows failed.
- Each event ends with the row's error, for example
A row failed to write. Error: relation "orders" does not exist. The error is left off when the message already includes{{row.dl_error}}. - Long errors are shortened to 300 characters. Secret values in the error text are masked.
- After the events, one summary event is recorded per distinct error, with its count and the node that failed:
200 rows failed at Snowflake Target: Target node "Snowflake Target" is missing DB schema.A code from the database or service shows in brackets. - Summary events follow the node's level, emission trigger and the run's Min log level. They count only the rows that produced an event.
- To leave the error out, add
dl_errorto Extra redaction keys.
Failed rows carry these fields, which the message and fields can also use:
| Field | Value |
|---|---|
dl_error | The error message. |
dl_error_code | The database or service error code. Present only when the error has one. |
dl_failed_node_id | The id of the node that failed. |
dl_failed_node_name | The name of the node that failed. |
When a whole node fails, for example because a setting is missing, every row carries the same error and the run log states it once beside the row count.
Examples
Check what a Compute node receives.
Level info, message Invoice {{row.invoiceId}} total {{row.amount}}, Emission trigger When expression is truthy with {{row.amount}} > 1000. Only large invoices are logged, and every row continues to the Compute node.
Use a condition on large runs: ten matching events are easier to read than ten thousand.
Troubleshooting
| Symptom | Cause | What to do |
|---|---|---|
| No events appear | The node's level is below the run's Min log level, or the condition never matched. | Lower Min log level at launch, or check the expression. |
A value shows as {{row.field}} or empty | The field name differs from the upstream field. | Use Insert field to pick the exact name. |
| A secret appears in the fields | Its field name is not in the redaction lists. | Add the name to Extra redaction keys, or do not log that field. |
| A number is recorded as text | The value mixes text and a token. | Use only {{row.field}} in the value. |
| Every failed row shows the same error | The upstream node failed as a whole, not row by row. | Fix the setting the summary event names, then run again. |