Skip to main content

Log node

Output & Observability

What it does​

The Log node records a structured event for each row, or for the rows that match a condition: a level, a message and a set of fields. The events appear in the run's Log Events card. Rows pass through unchanged, so the node observes the flow without changing it.

Log node events are kept in the run log. To send run, error and audit events to an external logging service, see Manage Account > Log Destinations.

When to use​

  • You want to see what a node received, without adding a Load you later remove.
  • You want row values, such as a customer id or an order total, recorded with each event.
  • A flow logs at debug while you build it, and you want those events dropped in scheduled runs.

Configure​

Select the node and press Configure to open Configure Log. The steps are Name & level, Message, Fields, Advanced and Summary & preview.

Save and Done stay unavailable while a required setting is missing. Cancel discards the changes made since the dialog opened or was last saved. A message confirms each save.

  • Level: debug, info, warn, error or fatal.
  • Message: a template. {{row.field}}, {{exec.varName}} and {{sys.runId}} insert values. Insert field adds an upstream field at the cursor. An unclosed {{, an unknown namespace or an unknown sys key shows as an error. A field the upstream schema does not list shows as a warning. Neither blocks saving.
  • Fields: name and value pairs recorded with the event. Add a field from the upstream node adds a row with the field's name and {{row.field}}. A value that is only {{row.field}} keeps its type, so numbers stay numbers. Any other value is recorded as text.

Advanced

  • Emission trigger:
    • Every row (default).
    • When expression is truthy: the expression is a template, filled in per row. A comparison with >, >=, <, <=, =, ==, != or <> is evaluated; ordering comparisons are numeric. Any other text counts as true unless it is empty, false, 0 or no.
    • Only at or above run's minimum level.
  • Extra redaction keys: a comma-separated list of field names to hide, added to the standard list (password, token, secret, api key, bearer, authorization). Matching is exact and ignores case.

Min log level on the run launch sets the lowest level recorded for that run. The default is info, so debug events are dropped unless you lower it.

Ports​

PortDirectionKindNotes
inindataRows to log.
outoutdataEvery input row, unchanged.
erroroutcontrolReceives the input rows when the node fails.

Reading the events​

Open the run and find the Log Events card. Each event shows its level, the node's name and the message. Expand an event to see its fields, with redacted values hidden. Filter by level, or search the list.

Logging failed rows​

Connect a node's error port to a Log node to record why rows failed.

  • Each event ends with the row's error, for example A row failed to write. Error: relation "orders" does not exist. The error is left off when the message already includes {{row.dl_error}}.
  • Long errors are shortened to 300 characters. Secret values in the error text are masked.
  • After the events, one summary event is recorded per distinct error, with its count and the node that failed: 200 rows failed at Snowflake Target: Target node "Snowflake Target" is missing DB schema. A code from the database or service shows in brackets.
  • Summary events follow the node's level, emission trigger and the run's Min log level. They count only the rows that produced an event.
  • To leave the error out, add dl_error to Extra redaction keys.

Failed rows carry these fields, which the message and fields can also use:

FieldValue
dl_errorThe error message.
dl_error_codeThe database or service error code. Present only when the error has one.
dl_failed_node_idThe id of the node that failed.
dl_failed_node_nameThe name of the node that failed.

When a whole node fails, for example because a setting is missing, every row carries the same error and the run log states it once beside the row count.

Examples​

Check what a Compute node receives.

Level info, message Invoice {{row.invoiceId}} total {{row.amount}}, Emission trigger When expression is truthy with {{row.amount}} > 1000. Only large invoices are logged, and every row continues to the Compute node.

Use a condition on large runs: ten matching events are easier to read than ten thousand.

Troubleshooting​

SymptomCauseWhat to do
No events appearThe node's level is below the run's Min log level, or the condition never matched.Lower Min log level at launch, or check the expression.
A value shows as {{row.field}} or emptyThe field name differs from the upstream field.Use Insert field to pick the exact name.
A secret appears in the fieldsIts field name is not in the redaction lists.Add the name to Extra redaction keys, or do not log that field.
A number is recorded as textThe value mixes text and a token.Use only {{row.field}} in the value.
Every failed row shows the same errorThe upstream node failed as a whole, not row by row.Fix the setting the summary event names, then run again.