Example 3: MySQL -> S3 CSV export with Filter
Why this example
A retail operations team has an internal MySQL orders database. The finance team wants a daily CSV drop of high-value orders (amount > 100) in S3, partitioned by date, so the FP&A pipeline downstream can pick it up. They don't want all orders, and they want fewer columns than the source table.
Helix Beverages context: the legacy partner ordering portal Helix acquired in 2023 runs on MySQL and is slowly being migrated. This recipe is how Finance gets high-value orders out of that MySQL today, in the shape the FP&A team's existing S3 pipeline already expects (see the Helix overview for the wider migration story).
At a glance
- Connectors used: MySQL, AWS S3
- Nodes used: Start, Source, Filter, Reshape, Transform, Load
- Schedule: daily at 01:00 UTC (after the upstream nightly close)
- Direction: One-way (MySQL -> S3)
Canvas
Start
Source: MySQL.orders
Filter: amount > 100
Reshape: pick columns
Transform: format date
Load: S3 (CSV)
Step-by-step
| # | Node | Action | Key configuration | Output shape |
|---|---|---|---|---|
| 1 | Start | -- | -- | -- |
| 2 | Source | MySQL: query | SELECT order_id, customer_id, amount, currency, created_at, region, sales_rep_id, raw_payload FROM orders WHERE DATE(created_at) = CURDATE() - INTERVAL 1 DAY | Full order row |
| 3 | Filter | -- | Predicate: row.amount > 100. Use the KEPT port to continue downstream; rows that fail the predicate are dropped (or routed via the DROP port for audit). | Same shape, fewer rows |
| 4 | Reshape | -- | Project columns: order_id, customer_id, amount, currency, created_at, region. Drop raw_payload, sales_rep_id. | Narrower row |
| 5 | Transform | -- | Cast created_at to YYYY-MM-DD string; build a derived column partition_date for the S3 key. | CSV-ready row |
| 6 | Load | S3: Upload File | Bucket finance-exports, key orders/dt=\{\{row.partition_date\}\}/orders.csv, format CSV, compression gzip. Append rows per date partition. | -- |
Variations
- Different DB family -- swap the Source to PostgreSQL, MSSQL, Oracle, or Snowflake. The SQL dialect for the date predicate changes (
current_date - 1etc.) but the rest of the canvas is identical. - Multiple files instead of one -- replace the Filter + single Load with a Branch keyed on
regionand route each branch to a separate Load. - JSON instead of CSV -- change the Load's format to
jsonland drop the Transform node entirely; JSON serialisation preserves types. - Same export to two clouds -- Fork the row stream after Reshape; one branch -> S3, the other -> Azure Blob.