Example 3: MySQL -> S3 CSV export with Filter

Data Pipeline examples

Why this example

A retail operations team has an internal MySQL orders database. The finance team wants a daily CSV drop of high-value orders (amount > 100) in S3, partitioned by date, so the FP&A process downstream can pick it up. They don't want all orders, and they want fewer columns than the source table.

Helix Beverages context: the legacy partner ordering portal Helix acquired runs on MySQL and is being migrated. This recipe is how Finance gets high-value orders out of that MySQL database, in the shape the FP&A team's S3 process expects (see the Helix overview for the wider migration story).

At a glance

• Connectors used: MySQL, AWS S3
• Nodes used: Start, Source, Filter, Reshape, Transform, Load
• Schedule: daily at 01:00 UTC (after the upstream nightly close)
• Direction: One-way (MySQL -> S3)

Canvas

Start

Source: MySQL.orders

Filter: amount > 100

Reshape: pick columns

Transform: format date

Load: S3 (CSV)

Step-by-step

#	Node	Action	Key configuration	Output shape
1	Start	None	None	None
2	Source	MySQL: query	SELECT order_id, customer_id, amount, currency, created_at, region, sales_rep_id, raw_payload FROM orders WHERE DATE(created_at) = CURDATE() - INTERVAL 1 DAY	Full order row
3	Filter	None	Predicate: row.amount > 100. Use the KEPT port to continue downstream; rows that fail the predicate are dropped (or routed via the DROP port for audit).	Same shape, fewer rows
4	Reshape	None	Project columns: order_id, customer_id, amount, currency, created_at, region. Drop raw_payload, sales_rep_id.	Narrower row
5	Transform	None	Cast created_at to YYYY-MM-DD string; build a derived column partition_date for the S3 key.	CSV-ready row
6	Load	S3: Upload File	Bucket finance-exports, key orders/dt=\{\{row.partition_date\}\}/orders.csv, format CSV, compression gzip. Append rows per date partition.	None

Variations

• Different DB family: swap the Source to PostgreSQL, MSSQL, Oracle, or Snowflake. The SQL dialect for the date predicate changes (current_date - 1 etc.) but the rest of the canvas is identical.
• Multiple files instead of one: use a Router keyed on region and route each value to a separate Load.
• JSON instead of CSV: change the Load's format to jsonl and drop the Transform node entirely; JSON serialisation preserves types.
• Same export to two clouds: Fork the row stream after Reshape; one branch -> S3, the other -> Azure Blob.