Setting up Databricks

This guide will walk you through gathering the necessary connection details from Databricks and configuring SaddleData to write to your Delta Lake tables.

1. Generate a Personal Access Token

To allow SaddleData to authenticate with Databricks, you need a Personal Access Token.

1. Log in to your Databricks Workspace.
2. Click your username in the top right corner and select Settings (or User Settings).
3. Go to the Developer tab.
4. Next to Access tokens, click Manage.
5. Click Generate new token.
6. Enter a comment (e.g., "SaddleData Integration") and an optional expiration lifetime.
7. Click Generate.
8. Important: Copy the token immediately and store it securely. You won't be able to see it again.

2. Get Connection Details

You need the HTTP Path and Hostname of your SQL Warehouse or Cluster.

1. In the sidebar, click Compute.
2. Click on the SQL Warehouse or Cluster you want to use.
3. Scroll down to the Connection Details tab (or "Advanced Options" -> "JDBC/ODBC").
4. Copy the following values:
   • Server Hostname (Host)
   • Port (usually 443)
   • HTTP Path

3. Configure SaddleData

1. Log in to SaddleData.
2. Go to Connections > New Connection.
3. Select Databricks Connection.
4. Enter the details you gathered:
   • Host: Paste the Server Hostname.
   • Port: 443
   • HTTP Path: Paste the HTTP Path.
   • Personal Access Token: Paste the token you generated in Step 1.
   • Catalog: (Optional) Enter your Unity Catalog name if applicable (e.g., main).
   • Schema: (Optional) Enter the default schema (e.g., default or analytics).
5. Click Test Connection to verify everything is working.
6. Click Add Connection.

You are now ready to use Databricks as a destination in your Flows!