Transform node
The Transform node maps upstream row columns to target columns, derives new fields with expressions, casts types, and shapes rows into whatever a downstream Load or Message expects. It's the most-used node in the flow builder.
When to use
- The target expects a different column set than the source (rename, drop, add).
- You need to derive a field from other fields (
fullName = CONCAT(first_name, ' ', last_name),region = LOWER(country_code)). - You need type coercion between source and target (string date → ISO date, string number → integer).
Ports
| Port | Direction | Kind | Notes |
|---|---|---|---|
in | in | data | Rows to shape. |
out | out | data | Shaped rows. |
error | out | control | Fires on expression evaluation errors when the mode is strict. |
notify | out | control | Notification policy port. |
Configuration
- Mappings. An ordered list of
{ from, to, expr }rows.frompicks an upstream column;tois the output column name; optionalexpris a runtime expression (LOWER(email),CONCAT(first, ' ', last),IF(EQ(status, 'active'), 1, 0)). Whenexpris empty the value passes through unchanged. - Custom fields. Fields that don't correspond to any upstream column — pure derivations (
created_by = "system",ingested_at = NOW()). - Transform actions. Column-family actions like
LOWERCASE ALL,TRIM ALL,NULL EMPTY STRINGS, andCOERCE DATE FORMATthat apply after the field-level mappings. - Drop unmapped columns. When on, the emitted row contains only the mapped output columns. When off, unmapped upstream columns pass through unchanged.
Configure Action walkthrough
- Left column: every upstream column with its detected type.
- Right column: target columns with their expected type (inferred from the downstream node's schema when possible).
- Drag-drop or click to pair; the pair becomes a mapping row.
- Click a mapping row to open the MappingExpressionModal — a 3-pane editor with the fields / variables / caches picker, a textarea + live preview, and a function library grouped by category.
Runtime behaviour
- Expressions evaluate per row via the shared
evalTransformExpr. Function catalog:UPPER,LOWER,TRIM,CONCAT,SUBSTRING,INDEX_OF,SPLIT,REPLACE,LENGTH,COALESCE, comparisons (EQ,GT,LT,AND,OR,NOT,IF), arithmetic (ADD,SUB,MUL,DIV,ROUND), dates (NOW,TODAY,ADD_DAYS,ADD_MONTHS,PARSE_DATE,FORMAT_DATE), identity (UUID), env / cache (VAR,SYS,CACHE_KEY,CACHE_GET). Nested calls supported. - Row order preserved.
CACHE_*calls hit the run-scoped Cache node; use them to memoize an enrichment result.
Failure modes
- Expression parse error. Static; caught at Save time.
- Runtime evaluation error. Falls back to
nullfor the derived field when the expression throws on a specific row; the run continues. - Target column not in downstream schema. Not enforced by Transform — a Load with strict schema enforcement is the gate.