Join node
The Join node correlates rows from two upstreams on a key using standard SQL join semantics. Under the hood it's a hash-join: the smaller side is loaded into a hash table, the larger side is streamed against it. Composite keys and per-side expressions are supported for real-world matching (case normalization, trimming, concatenation).
When to use
- Two upstream streams share a key and you need the matching rows combined.
- The join must survive partial completion — Join waits for BOTH upstream Sources to finish before dispatching, so a slow side does not fire the join with only fast-side rows.
- The key isn't a plain field on either side and needs a preparatory transformation (
LOWER(email)on one side,TRIM(customer_id)on the other).
Ports
| Port | Direction | Kind | Cardinality | Notes |
|---|---|---|---|---|
left | in | data | one | The left upstream. |
right | in | data | one | The right upstream. |
out | out | data | one | Joined rows. |
error | out | control | one | Fires when the build-side row limit is exceeded. |
notify | out | control | one | Notification policy port. |
Configuration
- Join type. Inner (default), Left, Right, or Full outer. SegmentedControl in the Configure modal.
- Key expressions. Paired list — one row per key component. Each side accepts either a plain field name (
customerId) or a runtime function expression (LOWER(email),CONCAT(email, '_', region),TRIM(id)). Composite keys narrow the match to rows that agree on EVERY pair; sparsity in any pair blocks the match. - Output fields. Optional table controlling which fields land in the output row. Empty (default) emits the full
{ ...leftRow, ...rightRow }spread merge. Non-empty emits exactly the listed fields, each optionally renamed via therenamecolumn. When the modal opens with declared upstream schemas, this table is pre-populated with every left + right column checked; uncheck what you don't need. - Build side.
auto(default; picks the smaller side by row count),left, orright. Explicit override optimizes a known-imbalanced join. - Memory guard.
buildSideRowLimit(default 1 000 000). Exceeding it routes an error to theerrorport with the offending side and row count in the diagnostic.
Runtime behaviour
- Fan-in barrier. Join is a barrier node: when its reachable-source set has more than one entry, the runtime skips Join during every per-source pass and dispatches it exactly once, after every source has completed, with rows accumulated in a run-scoped buffer. That's what makes a two-Source join produce matched rows without seeing partial state.
- Composite key builder. Every key pair is evaluated per row via the shared
evalTransformExprand joined into a single hash string using the codebase's SOH separator convention. Missing components disqualify the row from matching (so a null on one field of a composite key does not accidentally collide with another row's null on the same field). - Output projection. When
outputFieldsis populated, each emitted row is materialized field-by-field from the raw left / right rows rather than the merged shape. Same-name collisions across sides can be resolved with an explicitrenamein the picker.
Failure modes
- Empty output on a two-Source flow. Should not happen after the 2026-07-13 fan-in barrier fix. Grep the run log for
MULTI-SOURCE DRAIN: presence + non-empty rowsOut means the barrier ran and the keys / column policy is the issue. Absence means the barrier didn't detect the shared-downstream topology — that's a bug worth surfacing. - Build-side row limit. The
errorport emits a diagnostic naming the trigger side and row count. Either raisebuildSideRowLimit, flipbuildSideto force the other side, or pre-filter upstream. - Wrong key values. Wrap both sides in
LOWER(...)/TRIM(...)etc. in the paired-list editor. The runtime expression engine supports the full function catalog (UPPER,LOWER,TRIM,CONCAT,SUBSTRING,IF,ADD,ROUND,ADD_DAYS, etc.) with nested calls.