Skip to main content

Meet Helix Beverages: a Data Pipeline Examples backdrop

Data Pipeline examples

The 10 Data Pipeline Examples share one fictitious company as a backdrop: Helix Beverages. Its systems, partners and pain points are drawn from real DataLug customer conversations. As you read each example, picture it landing on a team's desk at Helix.

Meet Helix Beverages​

Helix Beverages is a mid-market North-American craft beverage company. About 1,400 employees, three production plants (Vermont, Colorado, British Columbia), distribution across the US and Canada via roughly 60 independent distributors, and a small direct-to-consumer e-commerce channel that ships in 28 states. Revenue around 380M USD annually, growing 9 to 12 percent year on year. They are not big enough to staff a dedicated integration platform team, but big enough that the spreadsheets and ad-hoc Python scripts that run the business keep breaking.

The ecosystem​

Helix's reality is a textbook mid-market mix: a strong core ERP, a fast-growing CRM, partner systems that range from "modern API" to "SFTP only, no exceptions", and an analytics layer the leadership team genuinely uses.

Internal divisions​

DivisionPrimary systemsWhat they care about
SalesSalesforce (Account, Opportunity, Order, Case)Closing deals, accurate pipeline, account hierarchy
Customer SuccessSalesforce (Case, Account), MSSQL (Order history)First-response time, churn signal, support workload
MarketingGoogle Drive, Google Sheets, Dropbox (creative), HubSpotCampaign analytics, content velocity, agency hand-off
Finance and AccountingMSSQL (ERP), Oracle (legacy GL from 2019 acquisition)Month-end close, AR aging, audit trail
Operations and Supply ChainMSSQL (inventory), SFTP (distributors), AWS S3 (co-packer manifests)On-time delivery, stockouts, plant utilization
IT and SecurityMicrosoft 365 (SharePoint, OneDrive, Teams), Azure Blob (backups)Identity, change management, compliance
Exec and BISnowflake (warehouse), Google Sheets (KPIs), Power BI (boards)Weekly KPI review, board reporting

Strategic partners​

Partner typeExamplesProtocol they prefer
DistributorsEastCoast Bev Group, Pacific Trade, Northern Tides DistributionSFTP nightly drops of order, inventory, depletions files
Co-packersHudson Bottling, Cascade CanningAWS S3 prefix per facility; manifests as JSONL
Retail buyersA major national grocer, two regional chains, several restaurant groupsSharePoint document libraries for compliance docs, EDI through a third party for orders
Creative agenciesTwo boutique studios in Toronto and PortlandDropbox shared folders, occasional Google Drive
Ad networksMeta, Google Ads, TikTokAPI integrations via the Generic API connector
Cloud vendorsSalesforce, Snowflake, Microsoft 365, Google WorkspaceNative APIs
On-call toolingPagerDuty, SMTP for low-severity alertsWebhook (Generic API), SMTP

End customers​

ChannelTouchpoints they generate
Major retail chainsBuyer-portal orders, compliance documents, sell-through reports
Restaurants and barsOn-premise orders through distributors, occasional direct chain accounts
Direct-to-consumerE-commerce orders (PostgreSQL event store), support cases, marketing opt-ins
Trade customersSample requests, tasting-event RSVPs, co-marketing brief drops

Why integration is hard for Helix​

Five named problems keep coming up in their integration planning meetings. Each one is the reason at least one of the 10 examples exists.

  1. CRM and ERP drift. Sales reps edit Accounts and Contacts in Salesforce. Customer-success agents edit the same records, then a few hours later a billing-driven update from MSSQL flips fields back. By Friday nobody trusts either system.
  2. BI lag. Snowflake is the dashboard everyone reads, but it is loaded by an overnight batch. When a regional VP needs to know how Tuesday is going on Tuesday, the answer is always "let me get back to you tomorrow."
  3. Partner files arrive on partner schedules. SFTP distributor drops, S3 co-packer manifests, and Drive creative deliveries all land asynchronously. Operations needs them merged and reconciled before 7am local time without anyone watching over the process.
  4. Exec reporting lives in Google Sheets. The CEO reads the same shared Google Sheet every Monday morning, and no BI tool will replace it. The Sheet needs to be refreshed automatically from Snowflake.
  5. Critical events go unnoticed. A transaction that fails post-settlement, a co-packer manifest that does not arrive, a Salesforce Case that hits P0: ops on-call needs to find out within minutes, in the channel they actually watch.

The Data Pipeline Examples as a Helix capability map​

Each row below names a Helix problem and points at the example that solves it. The same example may show up against more than one need.

#ExampleHelix problem it addresses
1Salesforce to Snowflake nightly syncBI lag (problem 2): get Sales and CS dashboards into Snowflake by 06:00 local
2Salesforce and MSSQL contact syncCRM and ERP drift (problem 1): two-way Contact sync without losing the latest edit
3MySQL to S3 CSV export with filterMigrating the acquired partner ordering portal off MySQL into a normalised S3 + Snowflake stack
4SFTP CSV to PostgreSQL with computePartner schedules (problem 3): land distributor inventory drops into the ops PostgreSQL store
5Oracle ETL with DbScript prelude and cacheBringing the legacy Oracle GL into Snowflake so Finance does not need two sets of dashboards
6Google Drive to SharePoint file copy with forkCompliance documents from a national retailer need to land in both the IT SharePoint and the Marketing Drive
7PostgreSQL fan-out with filterCritical events (problem 5): e-commerce events routed to Snowflake, S3, and on-call
8Salesforce email digest with routerCustomer Success Monday-morning case digest, sliced by region and priority
9Snowflake to Google Sheets executive reportExec reporting (problem 4): keep the CEO's Monday Sheet auto-fresh
10MSSQL and Salesforce reconciliationCRM and ERP drift (problem 1), order ledger: nightly reconciliation across the two systems of record

Suggested reading order​

If you are new to DataLug, read the 10 examples in this order:

  1. Start with one-way batches: examples 1, 3, 4, 5, 9. Same shape: Source -> Transform -> Load, on a schedule.
  2. Then the file-hub patterns: example 6. Drive to SharePoint copy with Fork.
  3. Then the fan-out and routing patterns: examples 7 and 8. One Source feeds multiple targets through Filter or Router.
  4. Then the two-way patterns: examples 2 and 10. Two data pipelines called in order from a parent, or a reconciliation with Cache.

Each example page links back to this overview.