Filter node
What it does
The Filter node checks each row against a condition. Rows that match leave on KEPT; rows that do not leave on DROP. Every row leaves on one of the two, so connecting DROP gives you the other half without a second node.
When to use
- A Source returns more rows than later steps need. A Filter right after it keeps the rest of the flow small.
- Rows that match a condition need different handling from the rest.
- The condition is a set of AND and OR rules over row columns.
To copy every row to several branches, use a Fork. To split rows across more than two paths, use a Router.
Configure
Select the node and press Configure to open Configure Filter. Choose a Filter Mode. A new Filter starts in Rule Builder with one empty rule.
Save and Done stay unavailable while a required setting is missing. Cancel discards the changes made since the dialog opened or was last saved. A message confirms each save.
Rule Builder: groups of rules. Each rule has a Column, an Operator and a Value. Each group combines its rules with AND or OR, and Combine Groups joins the groups with AND or OR. To keep rows where one column equals a value, use one rule with Equals. An empty Value matches empty values.
Expression: one expression; rows where it is true are kept. For example AND(GT(Amount, 0), NEQ(Status, 'Closed')). Use this mode to compare two columns.
A Filter saved with a single Column and Equals value opens in Rule Builder as one Equals rule, and keeps the same rows.
| Operators | Value |
|---|---|
| Equals, Not Equals, Contains, Not Contains, Starts With, Ends With | Text or number |
| Greater Than, Greater/Equal, Less Than, Less/Equal | Number |
| In List, Not In List | Comma-separated values |
| Is Empty, Is Not Empty, Is Null, Is Not Null | None |
Each column control has an fx button. Use it to filter on an expression instead of a plain column, such as LOWER(email) or TRIM(id). The editor has the function library, the upstream fields and live checking. The expression is evaluated for each row, and its result is compared with the rule's Value. An expression over a column the row does not have gives an empty value.
Ports
| Port | Direction | Kind | Notes |
|---|---|---|---|
in | in | data | Rows to check. |
out | out | data | KEPT: rows that matched. |
dropped | out | data | DROP: rows that did not match. |
error | out | control | Receives the input rows when the node fails, including when an expression cannot be evaluated. |
Examples
Open orders over zero. Expression mode: AND(GT(Amount, 0), NEQ(Status, 'Closed')). KEPT goes to a Transform and a Load; DROP is left unconnected.
Send other rows for review. Rule Builder with one group: email Is Not Empty AND country In List US, CA, GB. KEPT continues the flow; DROP goes to a Notification.
Latest row per customer. After a Compute node that numbers rows as rn, Rule Builder with one rule: rn Equals 1.
Troubleshooting
| Symptom | Cause | What to do |
|---|---|---|
| No rows are kept | The value differs in case or spaces, or the column name is wrong. | Use fx with LOWER(...) or TRIM(...), and check the column. |
| A number comparison gives odd results | The column holds text. | Text that looks like a number is compared as a number. Check for empty values and stray characters. |
| Rows disappear | DROP is not connected. | Connect DROP if you need those rows. |