Skip to main content

Connections

App page

A connection holds the address and credentials DataLug uses to reach one of your systems. Every connection is available to every project.

Where to find it​

Select Connections in the main menu. On the canvas, the Connection dropdown of a Source, Load or Database Script node also opens the connection form.

What you see​

The page has two views, switched in the top right of the card:

  • Connections: your configured connections.
  • Connectors: the connector catalog.

The connections list​

  • Each row says how many projects use the connection. Select the count to see the first three by name, the number of data pipelines each one runs, and a link to the full list. Check it before you change credentials or delete a connection.
  • Each environment has a chip on the row with its newest test result: a check for passed, a cross for failed or did not finish, a warning triangle for passed with a warning, and a dashed circle for not tested yet. Hover over a chip to see when it was tested. On a phone the chips sit on the line under the name.
  • The row menu has an entry per environment, such as Open DEV environment, which opens the Environments page on that environment.
  • Some connections show a project name, which opens that project.
  • Test history in the row menu shows the last three results for each environment. See Connection health checks.
  • The need attention button beside the search box, such as 3 need attention, lists only the active connections with an environment that failed, did not finish, or passed with a warning.
The row readsWhat it means
3 projectsThree projects use this connection. Select it for their names.
Not usedNothing uses it, so changing or deleting it affects no run.
UnknownThe count could not be read. Reload before you delete anything.

Test a connection from the list​

  • Test on a row tests every environment the connection has, one after another. Each chip shows its result as its environment finishes. A connection with no environments tests its base settings, shown as Base.
  • Select a chip to test that environment only.
  • While a test runs, the row shows Testing DEV... for the environment in progress.
  • When an environment does not pass, one line under the row says what went wrong, for example Some connection details are missing for QA, with a button to fix it. The button opens the connection on its Environments step with that environment open. Dismiss at the end of the line hides it until the next test result.
  • Details on that line says what to do, which environment failed, and when and by whom it was tested. Show technical response shows the system's own answer, and Copy copies it.
  • A failed test shows only on its row. Test moves on to the next environment after a failure, and an environment that does not answer within 70 seconds is marked failed with No answer from the test.
  • A Snowflake connection with no role, or with an administrator role, shows the same line in amber with Set a role.

Recently used​

A Recently used chip sits beside the search box once you have opened a connection. It narrows the list to the connections you opened most recently.

  • The count shows how many it will display.
  • It applies on top of the search box and the project you are in.
  • It switches itself off if a search leaves it with nothing to show.
  • The list stays grouped by connector family while the chip is on.

Row menu​

Each connection row has a menu:

  • Edit connection: rename the connection.
  • Configure: change its settings.
  • Duplicate: copy it as a new connection.
  • Export: download an encrypted copy.
  • View saved actions: list the saved actions on this connection.

Connector catalog​

The Connectors tab lists every connector with its status. Select one to open the New Connection wizard with that connector chosen. Each connector has its own page under Connectors.

What Test Connection reports​

A test lists each step it ran, under the result.

StepMeaning
Connection resolvedThe saved connection and the chosen environment were read.
Configuration validThe fields this connector needs are present.
Host reachableThe provider answered.
Credentials acceptedThe provider accepted the sign-in.
Data readableThe connection can read what it points at.

Each step shows one of four states:

  • Passed: the step succeeded.
  • Failed: the step did not succeed. The provider's own message is shown under it.
  • Skipped: the step does not apply, for example credentials on an SMTP relay that takes none.
  • Pending: the test stopped before this step. Nothing was sent.

A failed step followed by pending steps shows where to look. A missing password fails Configuration valid with Credentials accepted pending, so no credential was sent anywhere.

Some connectors report fewer steps. A driver that signs in and runs a query in one call cannot report the two separately.

Common tasks​

Create a connection​

  1. Select + New Connection
    On the Connections page.
  2. Page 1: choose the connector
    Search, or browse by category in the list on the left. Search matches the name, the category, the auth method and the operations a connector supports, so typing upload finds the file connectors. Press / to jump to the search box, the arrow keys to move, Enter to select and Esc to clear. Recently used lists the connectors you last made a connection with.
  3. Page 1: name the connection
    Choosing a connector suggests a name, for example My Salesforce Connection, after a prefix such as (Salesforce) -. Select the pencil to change it. Once you change the name it is yours, and picking a different connector updates the prefix without renaming it. If your account uses Organization Groups, Page 1 also asks which group the connection belongs to.
  4. Select Next
    Page 2 shows the settings for that system: basic fields, security (SSL/TLS and authentication) and advanced settings such as timeouts. Host, user and secret fields start empty. Greyed text in a field is an example of the value's shape, not a value.
  5. Add environment settings
    Optional. Add an environment to give DEV, QA and Production their own hosts or credentials.
  6. Test Connection
    Checks the settings for the selected environment. Some connectors need a runtime mapped to that environment to test.
  7. Save
    The connection appears in the list. If the save is refused, the dialog stays open with the reason above the buttons and nothing you typed is lost.

Nothing is selected on Page 1 until you choose it, and Next stays disabled until you do. Opening a connector from the Connectors tab, or following a link that names a system, pre-selects it. Select Change to pick another.

Set up a connection for each environment​

One connection can reach a different system in each environment, so the same data pipeline reads test data in DEV and live data in Production. You need at least two environments, a runtime mapped to each, and the host and credentials for each environment's system.

  1. Create the connection with the values most environments share, such as the port and the database name.
  2. Select Add Environment and pick DEV. Change the DEV host, user and password. Leave every other field as it is.
  3. Repeat for Production with the Production host and credentials.
  4. Pick DEV and select Test Connection, then pick Production and test again.
  5. Select Save.

How environment settings apply:

  • A new environment inherits every setting from the connection, including passwords, tokens and the connected account.
  • Each field shows the value the environment uses. Inherited values are marked Inherited from base.
  • Only the fields you change are saved for the environment. Use base value returns a field to the inherited value. An empty field also inherits.
  • An inherited password shows Inherited from base. Select Set for DEV to give the environment its own.
  • An OAuth connection uses the connection's account in every environment. Select Use a different account for DEV to connect another account for that environment.
  • Test Connection with an environment picked tests the settings that environment uses.
  • A run uses the settings of the environment it runs in. A job set to QA connects with the QA settings.
  • Environment names are matched without case.
  • Stored secrets are never sent back to the browser.

Create a connection from a node​

You can create a connection without leaving the canvas.

  1. Select a Source, Load or Database Script node.
  2. In the right panel, open the Connection dropdown and choose + New connection.... It is always the last entry.
  3. The wizard opens over the canvas, with the node's connector chosen where the node knows it.
  4. Select Save.

The new connection is selected on the node and joins the dropdown for every other node. Your canvas and any unsaved node settings stay as they were.

From the canvas, the wizard shows the full form for SQL Server, MySQL, PostgreSQL, Oracle, SFTP and SMTP. For other connectors it tells you to finish on the Connections page.

Once a node has a connection, Edit linked connection appears below the dropdown and opens that connection. Configuring an action on the node stores it on the connection, listed under View saved actions in the row menu.

Export and import​

Export downloads a connection as an encrypted file. Import, at the top of the Connections page, reads that file and creates the connection.

  • Passwords, tokens and other secrets are encrypted in the file.
  • An import fails when the file cannot be decrypted.
  • Review the imported connection's name before you save it.

Match environments. Environment overrides in the file are stored against your account's environments by name. When the file names an environment your account does not have, Import asks you to match each one to an environment, or to choose Do not import for it. A likely match, such as dev for Development, is selected for you. Import stays off until every row has a choice.

A connection with an override for an environment your account does not have shows Environment not found in its Environments section. Use Move to to attach the override to one of your environments.

Prepare the provider side​

  • File connectors (Google Drive, Google Sheets, OneDrive, SharePoint Files, Azure Blob, Dropbox, S3): prepare the OAuth app or access on the provider side first. See the File hub setup guide.
  • Salesforce: four authentication modes are available: OAuth Username/Password, OAuth Refresh Token, JWT Bearer and Hosted Salesforce Sign-In. Hosted sign-in uses a Salesforce Connected App. See Salesforce.

Schedule a health check​

A saved connection's settings end with a Health check section. Choose Daily or Weekly and a time to test the connection in every environment it has settings for. See Connection health checks.