# Databricks

To register a Databricks account into TrustLogix follow the below steps

1\. Create a data source entry in TrustLogix

2\. Run the auto generated notebooks in Databricks environment

3\. Gather all required information about Databricks account

4\. Update the details in TrustLogix and Save

#### **Create a** Databricks **Data Source In TrustLogix** <a href="#register-snowflake-data-source" id="register-snowflake-data-source"></a>

* Login to TrustLogix console and register a new Databricks data source from Menu → Data sources → Register new data source
* Provide the data source name, & click on Register data source
* Form will be refreshed with additional fields and script Download option

**Run the notebooks in** Databricks **environment**

* Click on Download button to download the notebooks
* Extract the downloaded Zip file which will contain **multiple** Python notebook files
* Ensure that each of the notebook prerequisites are performed before that particular notebook is run by providing all the arguments like the Account ID, Service Principal Name, Access Token etc.
* After successful run of each of the notebooks, required objects are created in Databricks
* Capture the output of the notebook and also gather&#x20;
  * Account ID&#x20;
  * Workspace Deployment Name
  * SQL Warehouse Identifier
  * Service Principal Name
  * Client ID
  * Client Secret

#### How to register additional workspaces to an existing Databricks datasource

The following are necessary to add additional workspaces to an existing Databricks datasource

1. TrustLogix Service Principal Id
2. TrustLogix Client Id
3. TrustLogix DP Service Principal Id(Only for trustlet/hybrid tenant)
4. TrustLogix DP Client Id(Only for trustlet/hybrid tenant)
5. Workspace URL e.g adb-180424056746427.6.azuredatabricks.net

Here, 1, 2 belong to the TrustLogix Control Plane Service Principal and 3, 4 belong to the DataPlane Service Principal. You can get the above from the Results section of the initial tlx\_databricks\_grants notebook used while registering Databricks account in the TrustLogix console.

**Step-1: Run following notebooks**

1. tlx\_databricks\_grants\_register\_workspace.ipynb
2. tlx\_databricks\_grants\_to\_cp\_service\_principal\_scheduler.ipynb(Only for trustlet/hybrid tenant)
3. tlx\_databricks\_workspace\_admin\_grant.ipynb

\#1 and #3 can be run from any workspace. #2 need to be run from new workspace to be registered.

> To execute the above notebooks in Databricks, it is suggested that the user or service principal must have a minimum of the following permissions:
>
> * **Workspace Admin**
> * **Metastore Admin**
>
> Account Admin permission not required.

**Step-2: Add the workspaces and warehouses in the TrustLogix console**

* Navigate back to TrustLogix data source form page
* Update following for the new workspace
  * Workspace Deployment Name(s)
  * SQL Warehouse Identifier(s)
* Click on save, system will validate the connection details and saves them
* If all details are valid, Databricks datasource should be updated successfully

**Step-3: Update secrets on AWS SSM (or Azure Key Vault) corresponding to the new workspace**

```
tlx-<tenant-id>-<account-id>-jdbcurl
tlx-<tenant-id>-<account-id>-warehouse
tlx-<tenant-id>-<account-id>-oauthEndpoint
```