Salesforce

CRM

The Salesforce connector queries and loads sObjects through the REST or Bulk API, moves files in and out of Salesforce, publishes Knowledge articles, and reads or ingests Data Cloud objects. It signs in with a connected app.

When to use it

• Salesforce is the system of record you read from or write to.
• You need Upsert on an external ID field, so a repeated run does not create duplicates.
• You load or extract large volumes with the Bulk API, chosen per action.
• You read unified profile data from Data Cloud, or ingest records into it.

What you need

• A connected app (or external client app) in Salesforce, with its Consumer Key as the Client ID and, for most modes, its Consumer Secret.
• The OAuth scopes your data pipeline needs. Data Cloud needs extra scopes, listed under Data Cloud objects.
• An integration user with permission on the objects the data pipeline reads and writes.
• For JWT, a certificate uploaded to the connected app, its private key in PEM form, and the integration user pre-authorised on the app.

Connect

The connection form lists its steps on the left. Open any step from that list. The Environments step holds the values that differ per environment.

Test connection is in the form footer. Beside it, Test against picks Base, the connection as saved, or an environment, whose values are applied first.

Auth Type is on the Authentication step and decides what else that step shows. Login URL is on the Connection step, and Remember Token is on Options.

Auth Type	Required before Save
Password	Login URL, Client ID, Username, Password
Refresh Token	Login URL, Client ID, Refresh Token
JWT (Private Key)	Login URL, Client ID, Username, Private Key
Hosted sign-in	Login URL, Client ID, Redirect URI, and a completed sign-in

A missing field is named under the form on every step.

Field	Step	What to enter
Auth Type	Authentication	Password, Refresh Token, JWT (Private Key) or Hosted sign-in.
Client ID	Authentication	The connected app's Consumer Key.
Client Secret	Authentication	The connected app's Consumer Secret. Not used by JWT.
Username	Authentication	Password and JWT. The integration user's username.
Password	Authentication	Password only.
Security Token	Authentication	Password only, when the org requires one.
Refresh Token	Authentication	Refresh Token only. A refresh token issued for the connected app.
Private Key	Authentication	JWT only. The PEM private key for the app's certificate.
Redirect URI	Authentication	Hosted sign-in only. Filled in. Add it to the connected app's callback URLs.
Connected Account, Display Name	Authentication	Hosted sign-in only. Filled in when you connect.
Login URL	Connection	https://login.salesforce.com, https://test.salesforce.com for a sandbox, or your My Domain URL.
Remember Token	Options	Keeps the access token between runs. On by default.

The Redirect URI is filled in for you. Copy it with the copy button and add it to the app registration exactly as shown. Use Edit beside the field only when the app is registered against a different address for DataLug.

Token lifetime

What DataLug stores	What it does
Access token	Used until it expires, then replaced.
Refresh token	Gets a new access token. A replacement is saved.
Expiry	Taken from Salesforce. A token with no expiry is not reused.

A connection with a refresh token renews itself. A connection with only an access token stops working when that token expires and needs a new sign-in.

When your connected app rotates refresh tokens, Salesforce issues a new one each time and retires the old one. DataLug saves the new one on the connection, and on the environment the run used.

One sign-in per environment

Hosted sign-in has a Connect button on the Authentication step and another in each environment on the Environments step. An environment can point at a different org, and each org grants its own token, so connect each environment you use. An environment's Connect uses the Client ID and Client Secret that environment resolves to, which may be the base connection's.

The base connection shows Connected only when it has its own token.

Supported operations

Action	Node	Operations
Query (Source)	Source	Query, by object and fields or as a Custom Query in SOQL
Load (Target)	Load	Insert, Update, Upsert, Delete, Merge, Publish, Unpublish
Query Data Cloud Object (Source)	Source	Query
Ingest to Data Cloud Object (Target)	Load	Ingest

API type on an action is REST or Bulk.

Delete is Soft delete, which sends records to the Recycle Bin, or Hard delete. Hard delete needs the Bulk API Hard Delete permission on the running user. It is not on the standard System Administrator profile, so grant it through a permission set or a cloned profile.

Org controls and write behaviour

A Salesforce node can turn validation rules, Apex triggers, flows, duplicate rules, matching rules and entitlement processes off or on around a node, and put them back after it. A Salesforce Target can also retry rejected rows, reprocess rows rejected with UNABLE_TO_LOCK_ROW, and group rows by parent record. See Execution hooks and policies.

Files and attachments

Salesforce keeps a file's bytes in a base64 field: VersionData on ContentVersion, Body on Attachment and Content on ContentNote. DataLug reads those fields from your org, so a custom base64 field works the same way.

Loading a file. Map a column holding the file to the base64 field. A column carrying a file from an earlier node is encoded for you. Each binary field in the Fields card has an encoding setting:

Setting	Behaviour
Encode when needed	Encodes a file from an earlier node. Passes a value through if it is already base64.
Always encode	Encodes, and fails the row if there is no file to encode.
Never encode	Passes the value through.

Inserting a ContentVersion returns ContentDocumentId on the row. Map it into a ContentDocumentLink in a later node to attach the file to a record. Set LinkedEntityId by record ID, or relate it by a key on the record, as described on the Transform node page.

Reading a file. Select the base64 field in the query. DataLug downloads each file and passes it to the next node, so a file connector can write it to S3, SharePoint, Dropbox or any other file target. Files are named from PathOnClient, then Title and FileExtension, then Name, and are removed when the run ends. A file that cannot be downloaded is reported in the run log and its row is skipped.

Notes. A note's body is HTML, and Salesforce accepts only bold, italic and underline in it. Map your text to Content and DataLug keeps those three styles, removes other formatting, keeps the text, and encodes special characters.

Attachment is the older object. Use Files for new work.

Email messages

• ParentId and RelatedToId can be set only while the message is a draft. The mapping sheet flags a row that sets one of them with another status.
• An email with no ParentId is private to the connecting user and does not appear in the Case activity history. The insert succeeds, and the mapping sheet warns about it.
• Recipients are on EmailMessageRelation. Load it as its own action.

Merging records

Merge combines duplicate Account, Contact, Lead, Person Account or Individual records. It appears in the operation list once the action names one of those objects. Set two columns on the action:

• Surviving record Id column. The record that is kept.
• Record Id columns to merge. Up to two records merged into it.

Both columns must hold Salesforce record IDs of 15 or 18 characters. External ID fields are not accepted. A row whose value is not a record ID is reported and not sent, and the other rows still run. Merge uses the SOAP API, whatever the action's API type. A merge cannot be undone.

Knowledge articles

Find your article types by typing kav in the object picker. Knowledge__kav is the standard one, and an org can have its own, such as FAQ__kav.

• Read. Query the article type like any other object. For data categories, switch the action to Custom Query and write WITH DATA CATEGORY yourself.
• Create. An Insert creates a draft, which customers cannot see until it is published.
• Translations. Rows in the same object, loaded or queried with the same action.
• Data categories. DataCategorySelection is its own object. Load it separately.

Publishing is a separate Load node:

Operation	What it does
Publish	Makes the draft live.
Unpublish	Takes the live article offline and leaves an editable draft.

Set Article Id column to KnowledgeArticleId. A draft insert puts that value on the row, so a Publish node after the draft load can map it across. Id is the article version's ID, which publishing does not accept.

• Publishing runs synchronously, so the run waits for each batch.
• Articles go in batches of 50. If one fails, every article in that batch is reported as failed and none of them went live. Re-run to retry.
• The connected user needs the Author Apex permission.

Data Cloud objects

Data Cloud keeps its own objects, which are not sObjects. A Data Model Object (DMO) or Data Lake Object (DLO) does not appear beside Account and Contact. Salesforce also calls Data Cloud Data 360, the name shown in Setup. One connection serves both kinds of object.

The connected app needs these five scopes. Without all five, reads can work and writes fail with a 403:

cdp_query_api
cdp_ingest_api
api
refresh_token
offline_access

DataLug exchanges your org token for a Data Cloud token. There is nothing to set for that step.

Querying a Data Cloud object

1. Add a Source node and select the Salesforce connection.
2. Open Configure Action.
3. Under the object list, choose Data Cloud objects.
4. Press Get Objects. DataLug lists the Data Model Objects.
5. Select one and choose the columns.

Switching between Salesforce objects and Data Cloud objects clears the object and its columns. With no columns chosen, the query selects every column. To read a Data Lake Object, set the action's entity type to DataLakeObject.

Ingesting into a Data Cloud object

Writing needs an Ingestion API connector, created in Data Cloud Setup. DataLug cannot list them, so you type its name.

1. In Salesforce, go to Data Cloud Setup > Ingestion API and note the connector name.
2. Add a Load node and select the Salesforce connection.
3. Open Configure Action and choose Data Cloud objects under the object list.
4. Select the object and set the operation to Ingest.
5. Enter the Ingestion API connector name.
6. Map your columns.

Ingest is the only write operation. DataLug checks every row before sending it, and a row Data Cloud would not accept is reported with the reason while the rest of the batch loads.

A row marked accepted has been taken by Data Cloud, not stored yet. Records become queryable after Data Cloud processes them, usually within a few minutes. dl_success is true and dl_note says so on every accepted row. Leave time between a write and a read of the same data, or run them on separate schedules.

Limits

• Salesforce meters API calls and concurrent requests per org and per edition. In Salesforce, see Setup > System Overview.
• A single file request carries up to 37.5 MB of encoded data, about 28 MB of file. A larger file fails its row with the measured size, and the rest of the batch loads.
• A Retry-After delay from Salesforce is honoured up to one minute.
• Bulk queries retry their paging after a 429.
• When the org has used its API request allowance (REQUEST_LIMIT_EXCEEDED), the run stops with that message instead of retrying.

DataLug retries a call that the provider refused with 429, and a call that could not connect. A read is also retried after a 408, a 5xx or a dropped connection. A write is retried only after a 429 or a failed connection.

A Retry-After delay of up to one minute is honoured. A longer one fails the call with the 429.

Troubleshooting

• Test connection passes and a Source finds no objects. The user can sign in but lacks object permissions. Grant read on the objects, then press Get Objects again.
• An Upsert creates duplicates. The external ID field is not set, or is not marked External ID in Salesforce. Without it, Upsert behaves as Insert.
• The connection stops working after a password change. The security token changes with the password. Update both on the connection.
• A run reports "The Salesforce sign-in has expired." The access token is past its expiry and there is no refresh token. Sign in again on the connection.
• A run reports an invalid session once, then succeeds. DataLug retries a rejected session once with a new token.
• A run in one environment updated another environment's token. That environment has no Salesforce values of its own, so the run used the base connection, and Salesforce replaced the base token.
• Publishing fails on the first batch. The connected user lacks Author Apex.
• A Data Cloud ingest fails with 403 and a query works. The connected app lacks cdp_ingest_api, which is separate from cdp_query_api.
• A run reports "Data Cloud refused the token exchange." The connected app has no Data Cloud scopes. Add the five listed under Data Cloud objects.
• A run reports "No Data Cloud source name is set on the action." The Ingestion API connector name is empty. Find it in Data Cloud Setup > Ingestion API.
• Data Cloud rows report accepted but a later query does not find them. Records take a few minutes to become queryable.
• A Data Cloud object you see in Setup is not in the list. The action lists the other kind. Switch the action's entity type between DataModelObject and DataLakeObject.
• A Data Cloud query returns values in the wrong columns. The account Owner emails DataLug support at support@datalug.io with the run ID.

Related

• Source node, Load node and Transform node.
• Marketing Cloud and HubSpot.
• Connections for environments and sharing.
• Execution hooks and policies for org controls and Salesforce Behavior.