Skip to main content

Jobs

App page

A Job deploys a data pipeline: it names a project, a data pipeline, an environment and, optionally, a schedule.

Where to find it​

Open Monitor > Jobs.

What you see​

The Jobs list has one row per job, with its schedule toggle and an Actions column.

  • Project names open the project.
  • Data Pipeline names open the project on that data pipeline's canvas.
  • Environment badges open the Environments page on that environment.

Legacy pipeline jobs​

Some accounts have jobs that run a pipeline instead of a data pipeline. These jobs keep running on their schedule and stay editable. On these jobs only, the Add Job form shows a Type field, and the name field reads Data Pipeline / Legacy Pipeline.

The name list for such a job shows the pipelines the project holds. A project with none shows an empty list. New jobs run a data pipeline.

Common tasks​

Create a job​

  1. Select Add Job
    On the Jobs page.
  2. Name the job and pick the Project, Data Pipeline and Environment
    The Data Pipeline list shows the data pipelines in the selected project.
  3. Check Deployment Readiness
    Lists the environment, runtime and connections the selected data pipeline needs. Only a missing environment or runtime blocks the job. Connections the data pipeline does not use do not block it.
  4. Pick the Run version
    Latest runs the current data pipeline. A saved version runs that saved snapshot.
  5. Set Scheduling (optional)
    A job can be saved without a schedule. To schedule it, pick a Mode: Daily, Hourly window, Weekly, Run once or Custom cron, then set the time and cadence.
  6. Save
    The job appears in the list. Each run, scheduled or started by hand, writes a Runtime Execution record.

Run a job now​

Select the play icon in the job row's Actions column. The run is queued, so it goes through the same path as a scheduled run and writes a run record, and the row reports "Run queued". The job's schedule does not change.

Running a job needs the job schedule permission, the same as editing a job.

Stop a running job​

While a job is running, select Stop Run on its row. A node that is mid-write may finish the rows in flight before the run stops.

Scheduled runs per month​

On the Free trial, schedules can start 50 runs per calendar month. The month starts on the 1st at 00:00 UTC. Paid plans have no monthly limit.

  • The Jobs page shows how many scheduled runs the account has used this month, for example "Scheduled runs this month: 48 of 50".
  • From 80% of the limit, a banner shows the usage and the reset date.
  • At the limit, scheduled runs are skipped until the reset date. Each skipped run appears in the run history as Skipped, with the reason "plan limit reached".
  • Run now still works at the limit. Runs started by Run now or by a webhook do not count toward it.

To remove the limit, upgrade the plan from the banner or from Billing & Usage.

Run history​

The run history shows the last 30 days. Change the Runs from and Runs to dates to see another range. Runs load 50 at a time; select Load more runs for the next ones.

The Version column shows which version ran: Latest, or the saved version number.

A run marked Rescheduled stopped at a connection's rate limit and runs again from its last checkpoint. It is not a failure. The pipeline's Start node decides whether a run suspends or fails; see Run performance.

Run a saved version​

A job with a saved Run version keeps running that snapshot until its Run version changes. Edits to the data pipeline do not reach the job. The project's canvas shows a notice while a job runs a saved version. To run the edits, set the job's Run version to Latest or to a newer saved version.

Changing a job's project or data pipeline sets its Run version back to Latest.

Turn a schedule off​

Use the schedule toggle on the job row. A job can be switched off even when the data pipeline it points at has been renamed or removed from the project.

Changing which data pipeline a job runs is checked: the data pipeline must exist in the selected project.