Dropbox
The Dropbox connector lists, reads and writes files in a Dropbox account. It signs in with your own Dropbox app.
When to use it
- The files you need are in Dropbox.
- A partner outside your organisation shares a Dropbox folder with you.
What you need
- A Dropbox app, created in the Dropbox App Console. Its app key and app secret are the Client ID and Client Secret.
- These four permissions enabled on the app's Permissions tab, saved with
Submit:
account_info.read,files.metadata.read,files.content.readandfiles.content.write. - The DataLug Redirect URI added to the app's redirect URIs on the Settings tab.
Enable the permissions before you connect. Dropbox refuses a sign-in that asks for a permission the app has not enabled.
Provider-side setup for this connector, including the app registration, the redirect URI and the permissions, is in the File hub setup guide. Connection basics and environments are on the Connections page.
Connect
The connection form lists its steps on the left. Open any step from that list. The Environments step holds the values that differ per environment.
Test connection is in the form footer. Beside it, Test against picks Base, the connection as saved, or an environment, whose values are applied first.
| Field | Step | What to enter |
|---|---|---|
| Client ID | Authentication | The Dropbox app key. |
| Redirect URI | Authentication | Filled in. Add it to the Dropbox app exactly as shown. |
| Client Secret | Authentication | The Dropbox app secret. |
| Connected Account, Display Name | Authentication | Filled in when you connect the Dropbox account. |
| Default Path Prefix | Connection | Optional. A folder that paths resolve under, such as exports/finance. |
| Notes | Options | Free text for your team. |
The Redirect URI is filled in for you. Copy it with the copy button and add it to the app registration exactly as shown. Use Edit beside the field only when the app is registered against a different address for DataLug.
Connect the Dropbox account on the Authentication step. Save stays disabled until the account is connected.
Supported operations
| Node | Operations |
|---|---|
| Source | List Files, Get File |
| Load | Upload File(s), Write Rows To File(s), Move/Rename, Delete, Create Folder |
Reading files. Recursive and Files only shape a listing.
Include (glob), Exclude (glob) and Regex (advanced) choose files by
name. Parse content into rows turns each CSV file into rows, with CSV has
header row and CSV Delimiter settings. Format Auto (by extension)
reads a .csv file as CSV.
Writing rows to files. Under Format & Behavior:
- Format is CSV, JSON or XLSX. Compress (.gz) compresses the file.
- File Name Template accepts the tokens
{runId},{YYYYMMDD},{HHmmss},{chunkIndex},{partIndex},{groupKey}and{field:MyField}. Append timestamp adds{YYYYMMDD}_{HHmmss}_{SSS}when the template has none. - Split is Single file, One per row or Group by. Group by takes one or more field names, separated by commas.
- Overwrite decides what happens when the file exists: Overwrite, Skip or Fail.
Choosing a path
The remote path field in Configure Action has a Browse button that lists the folder.
- Open enters a folder, and Up goes back. Use picks a file, and Use this folder picks the folder you are in.
- An operation that names a folder shows folders only.
- The field stays editable. Type a wildcard such as
/inbound/*.csv. - Browsing reads only. Nothing is written.
Limits
Dropbox rate limits depend on your Dropbox plan.
The API limits card on the connection form is collapsed by default. Every field on it is optional, and a blank field uses the default shown below it.
| Setting | What it does | Default |
|---|---|---|
| Requests per period, Period | Paces outgoing calls. Set both to throttle. | No throttle, or the rate shown on the card. |
| Page size | Rows per request. A larger value is reduced to the provider's maximum. | The maximum shown on the card, or set by the provider. |
| Max attempts | Tries per call, including the first. | 3 |
| Cool-off (ms) | The first wait after a 429. Later waits grow from it. | 500 ms |
| Requests at once | Requests open at the same time within one run. | No limit |
A Retry-After header from the provider overrides the cool-off. The pace
applies per connection and is shared by every run on it. See
Run performance.
DataLug retries a call that the provider refused with 429, and a call that
could not connect. A read is also retried after a 408, a 5xx or a dropped
connection. A write is retried only after a 429 or a failed connection.
A Retry-After delay of up to one minute is honoured. A longer one fails the
call with the 429.
Troubleshooting
- Test connection passes and a Source lists nothing. Check the path. A folder that does not exist lists as empty, not as an error. Pick the folder with Browse.
- A file is written but does not appear where you expect. Check the target path on the Load node. A path that resolves to another folder still succeeds.
- Large files. Staging space is limited per run, and the run log names any node that spilled to disk. See Files and staging.
- A connection that worked stops working. The stored authorisation was withdrawn or has expired. Open the connection and connect the account again. Editing the other fields does not restore it.
- A run or the path browser reports
missing_scope. The token lacks a permission the action needs. Enable the four permissions listed under What you need, then connect the account again. A token does not gain a permission when it refreshes. - Dropbox shows an invalid redirect URI when you connect. Add the Redirect URI to the app's Settings tab exactly as shown.
Related
- Source node and Load node.
- Google Drive and OneDrive.