Datadog
The Datadog log destination sends this account's data pipeline logs to your own Datadog organisation, through Datadog's logs intake for the site you choose.
When to use it
- Your team watches Datadog and wants DataLug run failures in the same place.
- You alert on connector errors with the monitors you already have.
What you need
- A Datadog organisation with Log Management.
- A Datadog API key. In Datadog, go to Organization Settings > API Keys.
- The Datadog site your organisation is on, such as
us1oreu1.
Connect
A log destination is set up under Manage Account > Log Destinations, not on the canvas. Press New destination, choose the provider, and fill in its fields. Log Destinations is available on the Teams and Business plans.
Each destination sends the streams you choose:
| Stream | Selected by default |
|---|---|
| Data pipeline runs | Yes |
| Node execution | Yes |
| Connector errors | Yes |
| Platform errors | No |
| Audit trail | No |
| Milo usage | No |
Minimum severity is debug, info, warn, error or fatal. Events below
it are not sent.
The audit trail does not include the request IP address. Milo usage carries token counts and cost, never prompt or response text.
Each event carries a unique eventId. Delivery is at least once, so use
eventId to remove duplicates.
Secrets are stored encrypted and are never sent back to the browser. To change one, enter a new value.
Send test event sends one event to the provider. A destination saves only after a test event succeeds, and a changed setting needs a new test.
DataLug sends new events about once a minute. While the provider is unreachable, events wait in DataLug and are sent when it recovers.
The destination list shows each destination's Status, Last delivery or last error, and two counts:
- Dropped counts events in a batch the provider rejected outright, such as
with a
400. Those events are not sent again. - Refused counts events the provider rejected one by one inside an accepted batch.
| Field | What to enter |
|---|---|
| Name | A name for this destination. |
| Datadog site | us1, us3, us5, eu1, ap1, ap2, uk1, us1-fed or us2-fed. There is no default. |
| API key | The Datadog API key. |
Your logs are stored in the region of the site you choose.
What is sent
Each event arrives with ddsource set to datalug and status set to the
event severity. These identity fields are sent as tags: env, account,
workflow, run and event_type.
Limits
- At most 1000 events or 5 MB per request.
- Long messages and tag values are shortened to fit Datadog's limits.
Datadog bills log ingestion and indexing to your Datadog account.
Troubleshooting
- The test passes but nothing appears in Datadog. Check the site. A key from a US organisation sent to the EU site authenticates and then indexes nowhere you are looking.
- The destination shows Failing. Datadog answered 401 or 403. The key was revoked or rotated. Enter a new one.
- Dropped is climbing. Datadog rejected a batch outright, usually with a 400. The Last delivery column shows the error.
- A message looks cut short. It was longer than Datadog accepts and was shortened to fit.
Related
- Grafana Loki, Elasticsearch and Splunk.
- Account for the Manage Account tabs.