Skip to main content

Sub-workflow node

Transform

The Sub-workflow node calls another workflow as a step inside the current one. It's the "call a function" primitive of the flow builder: shared normalization pipelines, enrichment chains, and export sinks all become reusable building blocks that any parent workflow can invoke.

When to use

  • You have a pipeline you want to reuse across projects (a normalization sequence, a compliance-check chain, a common enrichment path).
  • You want to keep the parent canvas readable — hoisting the detail of a substep into its own child workflow makes the parent easier to review.
  • You need the child pipeline's execution to inherit the parent's identity: same runId, same run environment, same quota bucket.

Ports

PortDirectionKindNotes
inindataParent rows handed to the child.
outoutdataRows the child's terminal node emitted.
erroroutcontrolFires when the child throws or the depth guard trips.
notifyoutcontrolNotification policy port.

Configuration

  • Child workflow — dropdown of every project visible in the current account except the parent project itself (self-reference is a design-time footgun the dropdown blocks even though the depth guard would catch it at run time).
  • Terminal node — which of the child's nodes emits the return rows for the parent. Free-text field (e.g. n_final_load); populate it with the node id from the child workflow's canvas.
  • Input binding.
    • passRows (default) — the parent's upstream rows are handed to the child as its input.
    • ignore — the child runs from its own Sources; the parent's upstream rows are discarded.
  • Max depth (default 3) — recursion guard. Protects against a workflow indirectly referencing itself through a chain of Sub-workflows. Exceeding the limit routes to the error port.

Runtime behaviour

  • The runtime constructs a nested RuntimeWorker in-process and calls execute() on the child manifest. No new process, no network hop.
  • The nested run inherits the parent's runId, runEnvironment, env-var block (DLR_PLAN_ID, DLR_RECORDS_PER_RUN, etc.), and ctx.logger (with a [sub:<node.id>] prefix on child log lines so the source is clear when a child logs to the parent stream).
  • Child manifest resolution goes through the backend's subWorkflowResolveService, which rejects cross-account references with 403 — same-account only in v1.
  • Failure propagation: the child throws → the Sub-workflow node's error port fires with err.subWorkflowContext populated.

Failure modes

  • Cross-account project id. Manifest resolve returns 403; error carries the reason. Sub-workflow references are same-account only.
  • Depth exceeded. Parent → child → grandchild is the default. If your composition needs deeper, raise maxDepth in the Configure modal, or (better) refactor to avoid the recursion.
  • Child terminal-node id doesn't exist. Failing fast: the run stops with Sub-workflow "X": terminal node "Y" is not present in the child manifest. Copy the correct node id from the child's canvas.