Receive a webhook from any system
A webhook starts a data pipeline when another system sends it a request. This guide follows one order from a shop to a data pipeline: the body format, the signature, duplicates, and turning the order's line items into rows.
The examples use one order with three line items. The Webhook Trigger node page covers the node itself.
Senders with their own page
Each of these has a setting under Signature and its own setup steps.
| Sender | Signature setting | Page |
|---|---|---|
| Jotform | Jotform (unsigned) | Jotform |
| Webflow | Webflow API webhook, or Webflow dashboard webhook (unsigned) | Webflow |
| Typeform | Typeform | Typeform |
| Calendly | Calendly | Calendly |
| HubSpot | HubSpot app webhook, HubSpot private or legacy app (v1), or HubSpot workflow webhook | HubSpot |
| Google Forms | DataLug signature, sent by an Apps Script | Google Forms |
| Meta Lead Ads, without the preset | Custom HMAC with a verification challenge | Meta Lead Ads with a generic webhook |
| Meta Lead Ads | Meta Lead Ads, or Meta Lead Ads (DataLug app) | Meta Lead Ads |
Any other sender uses the DataLug signature, Custom HMAC, Secret in header or an unsigned URL, below.
Body formats
The Content-Type of the request decides how the body is read.
| Content-Type | Becomes |
|---|---|
application/json | The JSON as sent. |
application/x-www-form-urlencoded | One object of the form fields. |
multipart/form-data | One object of the text fields. File uploads are refused. |
application/xml or text/xml | One object, with the root element as its top key. |
text/plain | { "body": "<the text>" } |
Any other type is refused with 415. An empty body is an empty object. Bodies are limited to 1 MB.
The same order as JSON and as XML
{"order":{"id":"ord_1001","total_price":"84.50","customer":{"email":"buyer@example.com"},
"line_items":[{"sku":"SKU-RED-S","quantity":2},{"sku":"SKU-BLU-M","quantity":1},{"sku":"SKU-MUG-01","quantity":1}]}}
<order>
<id>ord_1001</id>
<total_price>84.50</total_price>
<customer><email>buyer@example.com</email></customer>
<line_items>
<line_item><sku>SKU-RED-S</sku><quantity>2</quantity></line_item>
<line_item><sku>SKU-BLU-M</sku><quantity>1</quantity></line_item>
<line_item><sku>SKU-MUG-01</sku><quantity>1</quantity></line_item>
</line_items>
</order>
Both arrive as the same object, with order.line_items as a list of three items.
Form and XML values are text. "quantity": 2 in JSON is "2" from a form or XML body. Convert
a value to a number in a later node when you need one.
Form fields with brackets are nested: order[line_items][0][sku]=SKU-RED-S becomes
order.line_items[0].sku. A repeated field, such as tag=a&tag=b, becomes a list.
XML rules
- Repeated elements become a list.
- A wrapper whose children are all one element becomes that list, so
<line_items>is a list even with one<line_item>. - Attributes are kept as
@name. Text beside attributes is kept as#text. - XML with a
DOCTYPEorENTITYdeclaration is refused.
Signatures
Each webhook verifies requests with the setting chosen under Signature, in Manage Account > Webhooks, when you create the webhook or later under Settings. The senders listed above have their own settings; the general ones are below.
DataLug signature is for a sender you control. The sender signs with the secret DataLug generates; the Webhook Trigger node page shows the header.
Custom HMAC is for a system that signs its webhooks in its own way. Enter what the sender documents:
| Field | What to enter |
|---|---|
| Signature header | The header that carries the signature. |
| Signature prefix | Text before the signature, such as sha256=. Blank if none. |
| Algorithm | SHA-256 or SHA-512. |
| Encoding | Hex or Base64. |
| Signed content | The body alone, or a timestamp, a dot and then the body. |
| Timestamp header and tolerance | Only when a timestamp is signed. The tolerance is 1 to 3600 seconds. Older requests are refused, and so are requests more than 5 minutes ahead. |
| Reply status | 202 Accepted by default. Choose 200 OK for a sender that expects it. |
| Signing secret | The secret the sender gave you. |
Signature header X-Shopify-Hmac-Sha256, Algorithm SHA-256, Encoding Base64, Signed content Body,
and the signing secret from your Shopify app.
A secret you enter is stored encrypted and is never shown again. To change it, open Settings, enter the new secret in New signing secret and press Save settings. The old secret keeps working for 24 hours.
A request that fails verification is answered 401 and recorded as rejected, without its body.
Secret in header is for a sender that cannot sign but can send a fixed header, such as an API client or a tool like Postman. The sender sends the webhook secret itself in a header with every request. The DataLug signature is stronger: it never sends the secret.
| Field | What to enter |
|---|---|
| Secret header | Blank for Authorization, sent as Authorization: Bearer <secret>. Any other header, such as X-Api-Key, carries the secret as it is. |
| Signing secret | Blank to generate one, or the secret the sender already uses: 16 to 512 characters. |
Headers that a proxy or DataLug sets itself, such as Cookie, Host, Content-Type,
User-Agent and any X-Forwarded- header, cannot be used. The setup steps on the webhook show the
exact header to send. A generated secret is shown once, when the webhook is created or rotated.
Copy the webhook URL from Manage Account > Webhooks and use it in place of <webhook-url>.
curl -X POST "<webhook-url>" -H "Authorization: Bearer <secret>" -H "Content-Type: application/json" -d '{"order": {"id": "ord_1001"}}'
With a custom header, send -H "X-Api-Key: <secret>" instead. In Postman, choose Authorization >
Bearer Token and paste the secret, or add the custom header under Headers.
The secret header is never stored with a delivery and never shown in the app. A request with a
wrong or missing secret is answered 401. After a rotation the old secret keeps working for 24
hours, or until you press Revoke old secret.
Unsigned URL is for a sender that cannot sign. The webhook URL ends in a long random token, and
the URL itself is the credential: anyone who has it can start a run. The full URL is shown once,
when the webhook is created or rotated, and the webhook carries an Unsigned badge. Rotate
gives a new URL; the old one keeps working for 24 hours, or until you press Revoke old secret. A
request with a wrong or missing token is answered 404 and is not recorded.
Verification challenge
Some senders check a URL before they send to it: they call it with a GET request that carries a verify token and a challenge, and expect the challenge back. The DataLug signature, Custom HMAC, Secret in header and Unsigned URL settings can answer this. The Meta Lead Ads settings answer Meta's check on their own.
Under Settings or when creating the webhook, select Answer a GET verification challenge:
| Field | What to enter |
|---|---|
| Token parameter | The query parameter that carries the verify token. Blank for hub.verify_token. |
| Challenge parameter | The query parameter that carries the challenge. Blank for hub.challenge. |
| Mode parameter (Optional) | A parameter that must hold a set value, such as hub.mode. |
| Mode value | The value it must hold. Blank for subscribe. |
| Challenge verify token (Optional) | The token the sender sends: 16 to 256 characters with no spaces. Blank to generate one. |
A generated verify token is shown once, with a copy button. Paste it into the sender. When the
token matches, DataLug answers 200 with the challenge as plain text; otherwise 403.
For Secret in header, the GET must also include the secret header, exactly as a delivery does.
Without it, or with a wrong secret, DataLug answers 404, as for an unknown webhook. During the 24
hours after a secret rotation, the previous secret also works.
The verify token is stored encrypted and is never shown again. Rotate verify token on the webhook's row generates a new one, shown once; the old one keeps working for 24 hours, or until you press Revoke old verify token. Clearing the setting turns the check off and removes the token.
The webhook's row shows whether a sender has passed the challenge:
| Status | Meaning |
|---|---|
| Challenge passed | A sender answered with the right verify token, at the time shown. |
| Challenge passed with the previous verify token | The sender still uses the old token. Update the sender before the old token stops working. |
| Challenge not passed yet | No sender has passed the challenge. |
When a challenge was refused, the row also shows the last refusal, its time and its reason, such as Verify token did not match or Mode value did not match. A refusal does not clear an earlier pass, because anyone with the URL can send a GET request. Rotating the verify token or changing the challenge settings clears both.
Responses
| Signature | Accepted |
|---|---|
| DataLug signature | 202 |
| Custom HMAC | 202, or 200 when Reply status is 200 OK |
| Secret in header | 202 |
| Unsigned URL and every sender with its own page | 200 |
Which data pipeline runs
A delivery runs every data pipeline in the webhook's project with a saved Webhook Trigger node that uses the webhook. When none does, no run starts and the delivery shows Not used by any pipeline under Deliveries. The sender gets the reply above either way.
To run it, add a Webhook Trigger node that uses the webhook, save the data pipeline, then press Replay on the delivery.
Duplicates
Senders retry, so the same event can arrive more than once. A repeat starts no second run: it is
answered with the same status as the first delivery, the first delivery's id and
"duplicate": true. This holds when the first delivery was Not used by any pipeline: replay
that delivery instead.
With an event id. Set Event id for duplicates to a header or a field path in the body,
such as the header X-Shopify-Webhook-Id or the path order.id. Two requests with the same event
id are one delivery, even when the bodies differ, at any time.
Without one. A request whose body is identical, byte for byte, to one received in the last 24 hours is a duplicate. The same body after 24 hours runs again. When the configured event id is missing from a request, this rule applies to it.
One row per line item
By default the order is one row. To get one row per line item, select the Webhook Trigger node, press
Configure, set Rows from each delivery to Split array at path, and enter
order.line_items.
The order above then gives three rows. Each row has the line item's own fields and the order's other
fields with the prefix parent_:
| sku | quantity | parent_id | parent_total_price | parent_customer |
|---|---|---|---|---|
| SKU-RED-S | 2 | ord_1001 | 84.50 | { "email": "buyer@example.com" } |
| SKU-BLU-M | 1 | ord_1001 | 84.50 | { "email": "buyer@example.com" } |
| SKU-MUG-01 | 1 | ord_1001 | 84.50 | { "email": "buyer@example.com" } |
- A nested object, such as the customer, is kept whole under its
parent_name. - An empty list gives no rows.
- A path that is not in the delivery fails the node, and the run log names the path.
Test it
Press Test on the webhook to send a request with the webhook's own signature settings, with the secret in its header for Secret in header, or to its full URL for an unsigned webhook. It starts a real run.